[Erp5-dev] ERP5 Release System - proposal

[Shrenik Bhura]

Hello,

It would be beneficial if this valuable discussion could be moved and 
continued at the Discussions section of the wiki.

I shall do it 24 hrs from now if all are fine with it. In case of any 
objections/thoughts please let me know prior to that.

Thanks,
SB

Yoshinori Okuji wrote:
> Hello Łukasz,
>
> On Thursday 27 September 2007 12:21, Łukasz Nowak wrote:
>   
>> We present our idea how to organise release management for ERP5. We
>> would like to know your opinion about this idea. Any constructive
>> comments as well as criticism are welcome. We are making this proposal
>> in good faith - to provide the best possible solution in implementing
>> and development of ERP5 system, to give as much as possible possibility
>> to arise community of skilled developers around such great project as
>> ERP5 is itself.
>>     
>
> First of all, I would like to thank you for writing this proposal. I highly 
> appreciate that you have made significant effort towards a better maintenance 
> process.
>
> Let me describe how Nexedi has been maintaining the code, and succeeding in 
> implementing customers' projects so far:
>
> - All the code base, including Business Templates, unit tests, functional 
> tests, and many other types of tools, required for generic ERP5, has been 
> always published on the subversion repository. No permission is required to 
> access the source code at any time. Everything is immediately open to anybody 
> in the world freely. This way, more people can take part in the reviewing 
> process, and facilitating contribution. Also, developers recognize more 
> responsibility for the quality, because of the potential risk.
>
> - All the test results are sent to the public mailing list erp5-report every 
> day. People are free to check for the current status in the cutting-edge 
> development, thus it is easy to determine the latest stable revision from the 
> history of reports. If a result is good, but the system is still unstable, 
> this only means that we lack tests. Such tests should be complemented as soon 
> as possible, and contribution is very welcome.
>
> - Chanages are, in principle, peer-reviewed. Only trivial changes, such as 
> fixing a typo, do not require such a review. Some people are not allowed to 
> directly check in changes, when we still don't know the quality level of code 
> produced by them. Once a person proves that she has a good skill, we give 
> more permissions to her, so that we will not prevent her work. However, even 
> experienced developers, such as me and Jean-Paul, need a peer-review, before 
> committing a significant change. This way, although it may not be perfect, 
> the quality of software is kept acceptally good all the time. Note that I 
> have been using the same policy in another Free Software project for many 
> years successfully.
>
> - Backward compatibility is respected all the time. We break backward 
> compatibility, only if it is inevitable. Incompatibility is an evil, because 
> it produces too much work at a time when upgrading a system. We have been 
> publishing the information on incompatibility we made in the public wiki at 
> www.erp5.org, so that others will not get troubled. Still, we are always 
> trying to reduce incompatibility, and only deprecate old usage.
>
> - Code base used for a customer's project is taken out of the public 
> subversion repository, under the control by a project manager. A decision is 
> made mostly based on the status of unit tests and functional tests, but also 
> with reviwing recent changes of the code, because some kind of problems may 
> not be very visible only with automatic testing. Then, the code base is 
> upgraded from time to time (the frequency depends on the constraints in a 
> project) so that bug fixes and feature enchancement are incorporated. Again, 
> the decision of which revision should be taken is based on the same criteria 
> as above, as well as project conditions in some cases. This way, we can 
> implement a stable system without much trouble.
>
> As the technical leader of ERP5, I have been thinking of how to make releases 
> for some years, in a way that the maintenance cost would be negligible, and 
> woouldn't disturb development and contribution. The important questions are: 
> When and How. I consider that there is no question about "Why". As you 
> pointed out, releases make things more apparent and visible, so they are 
> definitely nice to have.
>
> Here are some of my ideas:
>
> - When
>
> In my understanding, there are three different ways to manage the timings of 
> making releases.
>
> One of them is feature-basis. When something significant is made, a new 
> release is published. This is often used in proprietary software products for 
> marketing purpose. Then, bug fixes are provided only in the form of patches. 
> In Open Source development, this does not make sense very much, since patches 
> can be included from the beginning. Instead, feature enhancement increments a 
> major version. For now, I am not talking about how to name versions, so I 
> don't take this way into consideration any more.
>
> Another is regularity-basis. If I understand correctly, this is the same as 
> your suggestion. In this scheme, releases are made in the same frequency, no 
> matter whatever is happening. The advantage is that it is guaranteed that the 
> realtime characteristics of releases against the head is quite good, that is, 
> the distribution of delays behind the head is minimized. However, this way 
> has a critical drawback that it does not reflect the reality of development. 
> Sometimes a significant amount of development is performed in a short term, 
> and sometimes not. As the nature of development, changes, in particular, 
> which address new problems, tend to make software less stable. I will discuss 
> this issue more below.
>
> The other is quality-basis. A new release is made when the head has good 
> quality of code. This is rather the reverse of the regularity-basis, so the 
> pros/cons are also reversed. This proceeds with the reality of development, 
> thus a delay can be potentially huge, but the quality of a release is well 
> maintained.
>
> Of course, in most projects, these ways are more or less mixed. Even if the 
> quality-basis approach is taken, it does not make sense to make a new release 
> every day, only because the quality is high. Even with the regularity-basis 
> one, if the quality cannot meet criteria on time, a delay must be inserted to 
> prevent catastrophy. Many factors should be taken into consideration to keep 
> good release management.
>
> As you might know, I have +10-year experience in many other Open Source/Free 
> Software projects. And, I have tried both the regularity-basis and 
> quality-basis ways. Overall, I have found that the quality-basis works far 
> better, because, for example, when a problem can be resolved appropriately by 
> a certain few people, and if they are not available for some reason for a 
> while, the regularity gets broken so easily. Enforcing a release quickly, 
> simply because of the policy, only produces unusable software.
>
> Also, I have found that release management is far from trivial. Of course, as 
> you suggested, we use the result of tests for making estimate on stability, 
> but tests may not cover everything. It is sometimes because not all tests are 
> implemented yet. It is sometimes because managers are not capable of 
> determining whether software is stable enough or not. Especially in the 
> preparation of a new release, bug fixes must be encouraged, but contributors 
> have their own time constraints, and may not always follow a release cycle. 
> In this case, managers must be able to define what bug fixes can be postponed 
> with the idea of possible danger, and address problems by themselves, 
> whenever necessary.
>
> Another perspective is how to facilitate people to contribute. Usually, 
> excellent developers want good feedback from managers. If they feel that 
> managers fully understand their changes, and evaluate the contribution 
> appropriately, they tend to help managers. Otherwise, they stop. This is not 
> related to whether they are paid or not. For instance, although workers at 
> Nexedi are paid, they are not always paid to contribute publicly. This part 
> really depends on their feeling and will. If they think that managers would 
> not understand their effort, they will just put changes locally and hide 
> them. This has been always the same as in volunteer-based projects, so I am 
> rather sure. Therefore, I believe that highly skilled persons must be 
> selected as release managers for ERP5. Otherwise, release management will 
> corrupt.
>
> - How
>
> There are two matters to consider: how many branches should be maintained or 
> developed, and how long a branch should be maintained.
>
> The simplest approach is to use a single trunk, and no other branch. This is 
> the current way. The benefit is that all the effort heads towards one place, 
> so the efficiency of development is very high. We do not need to worry about 
> forking. The drawback is that the quality can be random along the time, no 
> matter however much effort is put into stabilization.
>
> The semi-simple approach is, as you proposed, to use a main trunk, and one 
> branch to keep stability. And, to abandone older branches, once a new branch 
> is created. This is somehow easy and realistic. The benefit is that, without 
> having a lot of maintenance cost, stability is more or less ensured on a 
> branch. The drawback is that this way additionally requires porting changes 
> between a trunk and a branch, thus the development energy is scattered, and 
> extra effort and time are required for the maintenance.
>
> For now, I compare only these two. Again, I have experience with both 
> approaches in the past. In short, I met difficulty in the latter approach, 
> because the cost of porting patches was quite higher than I expected. After 
> all, I gave it up, and I chose a single trunk method.
>
> The way I did was the following: when I started to feel that I wanted to make 
> a new version, for example, because we had a good set of new functionality, I 
> forced people to stop writing a new feature, but encouraged people to test 
> the head, and fix it. In the meantime, all patches sent to me were waited, 
> and kept pending. Once I was satisified with the quality of code, I put a tag 
> and release it. Then, I restarted the normal cycle of development, and 
> integrating pending contribution.
>
> Actually, this was so effective. All contributors understood that this was 
> required, and accepted that patches were in a pending state. Besides stopping 
> people to modify the head brutally, no extra cost was raised.
>
> Surely, we must analyze if this way is effective for ERP5 as well, but I feel 
> that this is more appropriate.
>
> The reason is really the cost. Unfortuantely, nobody is a superman. I don't 
> think it is a bad thing to have a stable branch, but at the same time, I 
> doubt if it is feasible. If we use branching, another team for a branch would 
> be necessary to be formed, just like the security team in Debian, because the 
> backporting is surprisingly hard and time-consuming.
>
> If anybody wants to perform the backporting job, I would appreciate very much. 
> But be prepared that it is not trivial at all. One must be able to say what 
> is a pure fix, and must be able to segregate safe parts of patches from a 
> trunk, and adapt API usage, and so on. For now, I don't think this is 
> feasible with the current size of our community, but I would be very happy if 
> I am wrong.
>
> So, at the moment, my suggestion is to use a single trunk. We have proven that 
> writing tests for a good coverage makes projects successful, because tests 
> always notify us of the current status, and what must be fixed, without any 
> branching. Release managers should promote people to write more tests for 
> missing parts, and negotiate with developers when next release should be 
> made, so that others can collaborate.
>
> Now, this mail is getting too long, but let me describe some more additional 
> notes.
>
> - It is a very good idea to describe API changes. We already do upgrade notes, 
> but this is a nice complement.
>
> - Upgrade tests must be automated very well, otherwise it is too costly. Not 
> only unit tests, but also functional tests must be performed. Currently, the 
> number of functional tests is not good enough, in particular about 
> configuration-related stuff. Contribution is very welcome.
>
> - Release management may not be independent of development management. Both 
> are very tightly coupled, because releasing depends on the status of 
> development. Only experts can perform this kind of job, so that other 
> developes will listen to their advise.
>
> - I myself prefer co-maintenance, that is, having multiple maintainers. This 
> has a significant advantage in that even if one is not available (sick, busy, 
> or whatever), the project will not be stalled. I was proposed to be a 
> co-maintainer in a project, and it worked very well, so I'm proposing others 
> to do the same with me these days. I like it. For ERP5, 2-3 persons will be 
> good. Having too many is no good.
>
> I hope my message would be useful for further discussion. Please let me know 
> what you think.
>
> Regards,
> YO
>   

-- 
Shrenik Bhura, IntelliAnt CEO - Tel. +91(0)98310 33483
IntelliAnt: Consulting and Development of Free / Open Source Software
URL: http://www.intelliant.net