[Erp5-dev] ERP5 Release System - proposal

[Jean-Paul Smets]

Hi,

I created http://www.erp5.org/Discussion/ReleaseProcess
and will summarize discussion.

Regards,

JPS.

Shrenik Bhura a écrit :
> Hello,
>
> It would be beneficial if this valuable discussion could be moved and 
> continued at the Discussions section of the wiki.
>
> I shall do it 24 hrs from now if all are fine with it. In case of any 
> objections/thoughts please let me know prior to that.
>
> Thanks,
> SB
>
> Yoshinori Okuji wrote:
>   
>> Hello Łukasz,
>>
>> On Thursday 27 September 2007 12:21, Łukasz Nowak wrote:
>>   
>>     
>>> We present our idea how to organise release management for ERP5. We
>>> would like to know your opinion about this idea. Any constructive
>>> comments as well as criticism are welcome. We are making this proposal
>>> in good faith - to provide the best possible solution in implementing
>>> and development of ERP5 system, to give as much as possible possibility
>>> to arise community of skilled developers around such great project as
>>> ERP5 is itself.
>>>     
>>>       
>> First of all, I would like to thank you for writing this proposal. I highly 
>> appreciate that you have made significant effort towards a better maintenance 
>> process.
>>
>> Let me describe how Nexedi has been maintaining the code, and succeeding in 
>> implementing customers' projects so far:
>>
>> - All the code base, including Business Templates, unit tests, functional 
>> tests, and many other types of tools, required for generic ERP5, has been 
>> always published on the subversion repository. No permission is required to 
>> access the source code at any time. Everything is immediately open to anybody 
>> in the world freely. This way, more people can take part in the reviewing 
>> process, and facilitating contribution. Also, developers recognize more 
>> responsibility for the quality, because of the potential risk.
>>
>> - All the test results are sent to the public mailing list erp5-report every 
>> day. People are free to check for the current status in the cutting-edge 
>> development, thus it is easy to determine the latest stable revision from the 
>> history of reports. If a result is good, but the system is still unstable, 
>> this only means that we lack tests. Such tests should be complemented as soon 
>> as possible, and contribution is very welcome.
>>
>> - Chanages are, in principle, peer-reviewed. Only trivial changes, such as 
>> fixing a typo, do not require such a review. Some people are not allowed to 
>> directly check in changes, when we still don't know the quality level of code 
>> produced by them. Once a person proves that she has a good skill, we give 
>> more permissions to her, so that we will not prevent her work. However, even 
>> experienced developers, such as me and Jean-Paul, need a peer-review, before 
>> committing a significant change. This way, although it may not be perfect, 
>> the quality of software is kept acceptally good all the time. Note that I 
>> have been using the same policy in another Free Software project for many 
>> years successfully.
>>
>> - Backward compatibility is respected all the time. We break backward 
>> compatibility, only if it is inevitable. Incompatibility is an evil, because 
>> it produces too much work at a time when upgrading a system. We have been 
>> publishing the information on incompatibility we made in the public wiki at 
>> www.erp5.org, so that others will not get troubled. Still, we are always 
>> trying to reduce incompatibility, and only deprecate old usage.
>>
>> - Code base used for a customer's project is taken out of the public 
>> subversion repository, under the control by a project manager. A decision is 
>> made mostly based on the status of unit tests and functional tests, but also 
>> with reviwing recent changes of the code, because some kind of problems may 
>> not be very visible only with automatic testing. Then, the code base is 
>> upgraded from time to time (the frequency depends on the constraints in a 
>> project) so that bug fixes and feature enchancement are incorporated. Again, 
>> the decision of which revision should be taken is based on the same criteria 
>> as above, as well as project conditions in some cases. This way, we can 
>> implement a stable system without much trouble.
>>
>> As the technical leader of ERP5, I have been thinking of how to make releases 
>> for some years, in a way that the maintenance cost would be negligible, and 
>> woouldn't disturb development and contribution. The important questions are: 
>> When and How. I consider that there is no question about "Why". As you 
>> pointed out, releases make things more apparent and visible, so they are 
>> definitely nice to have.
>>
>> Here are some of my ideas:
>>
>> - When
>>
>> In my understanding, there are three different ways to manage the timings of 
>> making releases.
>>
>> One of them is feature-basis. When something significant is made, a new 
>> release is published. This is often used in proprietary software products for 
>> marketing purpose. Then, bug fixes are provided only in the form of patches. 
>> In Open Source development, this does not make sense very much, since patches 
>> can be included from the beginning. Instead, feature enhancement increments a 
>> major version. For now, I am not talking about how to name versions, so I 
>> don't take this way into consideration any more.
>>
>> Another is regularity-basis. If I understand correctly, this is the same as 
>> your suggestion. In this scheme, releases are made in the same frequency, no 
>> matter whatever is happening. The advantage is that it is guaranteed that the 
>> realtime characteristics of releases against the head is quite good, that is, 
>> the distribution of delays behind the head is minimized. However, this way 
>> has a critical drawback that it does not reflect the reality of development. 
>> Sometimes a significant amount of development is performed in a short term, 
>> and sometimes not. As the nature of development, changes, in particular, 
>> which address new problems, tend to make software less stable. I will discuss 
>> this issue more below.
>>
>> The other is quality-basis. A new release is made when the head has good 
>> quality of code. This is rather the reverse of the regularity-basis, so the 
>> pros/cons are also reversed. This proceeds with the reality of development, 
>> thus a delay can be potentially huge, but the quality of a release is well 
>> maintained.
>>
>> Of course, in most projects, these ways are more or less mixed. Even if the 
>> quality-basis approach is taken, it does not make sense to make a new release 
>> every day, only because the quality is high. Even with the regularity-basis 
>> one, if the quality cannot meet criteria on time, a delay must be inserted to 
>> prevent catastrophy. Many factors should be taken into consideration to keep 
>> good release management.
>>
>> As you might know, I have +10-year experience in many other Open Source/Free 
>> Software projects. And, I have tried both the regularity-basis and 
>> quality-basis ways. Overall, I have found that the quality-basis works far 
>> better, because, for example, when a problem can be resolved appropriately by 
>> a certain few people, and if they are not available for some reason for a 
>> while, the regularity gets broken so easily. Enforcing a release quickly, 
>> simply because of the policy, only produces unusable software.
>>
>> Also, I have found that release management is far from trivial. Of course, as 
>> you suggested, we use the result of tests for making estimate on stability, 
>> but tests may not cover everything. It is sometimes because not all tests are 
>> implemented yet. It is sometimes because managers are not capable of 
>> determining whether software is stable enough or not. Especially in the 
>> preparation of a new release, bug fixes must be encouraged, but contributors 
>> have their own time constraints, and may not always follow a release cycle. 
>> In this case, managers must be able to define what bug fixes can be postponed 
>> with the idea of possible danger, and address problems by themselves, 
>> whenever necessary.
>>
>> Another perspective is how to facilitate people to contribute. Usually, 
>> excellent developers want good feedback from managers. If they feel that 
>> managers fully understand their changes, and evaluate the contribution 
>> appropriately, they tend to help managers. Otherwise, they stop. This is not 
>> related to whether they are paid or not. For instance, although workers at 
>> Nexedi are paid, they are not always paid to contribute publicly. This part 
>> really depends on their feeling and will. If they think that managers would 
>> not understand their effort, they will just put changes locally and hide 
>> them. This has been always the same as in volunteer-based projects, so I am 
>> rather sure. Therefore, I believe that highly skilled persons must be 
>> selected as release managers for ERP5. Otherwise, release management will 
>> corrupt.
>>
>> - How
>>
>> There are two matters to consider: how many branches should be maintained or 
>> developed, and how long a branch should be maintained.
>>
>> The simplest approach is to use a single trunk, and no other branch. This is 
>> the current way. The benefit is that all the effort heads towards one place, 
>> so the efficiency of development is very high. We do not need to worry about 
>> forking. The drawback is that the quality can be random along the time, no 
>> matter however much effort is put into stabilization.
>>
>> The semi-simple approach is, as you proposed, to use a main trunk, and one 
>> branch to keep stability. And, to abandone older branches, once a new branch 
>> is created. This is somehow easy and realistic. The benefit is that, without 
>> having a lot of maintenance cost, stability is more or less ensured on a 
>> branch. The drawback is that this way additionally requires porting changes 
>> between a trunk and a branch, thus the development energy is scattered, and 
>> extra effort and time are required for the maintenance.
>>
>> For now, I compare only these two. Again, I have experience with both 
>> approaches in the past. In short, I met difficulty in the latter approach, 
>> because the cost of porting patches was quite higher than I expected. After 
>> all, I gave it up, and I chose a single trunk method.
>>
>> The way I did was the following: when I started to feel that I wanted to make 
>> a new version, for example, because we had a good set of new functionality, I 
>> forced people to stop writing a new feature, but encouraged people to test 
>> the head, and fix it. In the meantime, all patches sent to me were waited, 
>> and kept pending. Once I was satisified with the quality of code, I put a tag 
>> and release it. Then, I restarted the normal cycle of development, and 
>> integrating pending contribution.
>>
>> Actually, this was so effective. All contributors understood that this was 
>> required, and accepted that patches were in a pending state. Besides stopping 
>> people to modify the head brutally, no extra cost was raised.
>>
>> Surely, we must analyze if this way is effective for ERP5 as well, but I feel 
>> that this is more appropriate.
>>
>> The reason is really the cost. Unfortuantely, nobody is a superman. I don't 
>> think it is a bad thing to have a stable branch, but at the same time, I 
>> doubt if it is feasible. If we use branching, another team for a branch would 
>> be necessary to be formed, just like the security team in Debian, because the 
>> backporting is surprisingly hard and time-consuming.
>>
>> If anybody wants to perform the backporting job, I would appreciate very much. 
>> But be prepared that it is not trivial at all. One must be able to say what 
>> is a pure fix, and must be able to segregate safe parts of patches from a 
>> trunk, and adapt API usage, and so on. For now, I don't think this is 
>> feasible with the current size of our community, but I would be very happy if 
>> I am wrong.
>>
>> So, at the moment, my suggestion is to use a single trunk. We have proven that 
>> writing tests for a good coverage makes projects successful, because tests 
>> always notify us of the current status, and what must be fixed, without any 
>> branching. Release managers should promote people to write more tests for 
>> missing parts, and negotiate with developers when next release should be 
>> made, so that others can collaborate.
>>
>> Now, this mail is getting too long, but let me describe some more additional 
>> notes.
>>
>> - It is a very good idea to describe API changes. We already do upgrade notes, 
>> but this is a nice complement.
>>
>> - Upgrade tests must be automated very well, otherwise it is too costly. Not 
>> only unit tests, but also functional tests must be performed. Currently, the 
>> number of functional tests is not good enough, in particular about 
>> configuration-related stuff. Contribution is very welcome.
>>
>> - Release management may not be independent of development management. Both 
>> are very tightly coupled, because releasing depends on the status of 
>> development. Only experts can perform this kind of job, so that other 
>> developes will listen to their advise.
>>
>> - I myself prefer co-maintenance, that is, having multiple maintainers. This 
>> has a significant advantage in that even if one is not available (sick, busy, 
>> or whatever), the project will not be stalled. I was proposed to be a 
>> co-maintainer in a project, and it worked very well, so I'm proposing others 
>> to do the same with me these days. I like it. For ERP5, 2-3 persons will be 
>> good. Having too many is no good.
>>
>> I hope my message would be useful for further discussion. Please let me know 
>> what you think.
>>
>> Regards,
>> YO
>>   
>>     
>
>   


-- 
Jean-Paul Smets-Solanes, Nexedi CEO - Tel. +33(0)6 62 05 76 14
Nexedi: Consulting and Development of Libre / Open Source Software
http://www.nexedi.com
ERP5: Libre/ Open Source ERP Software for small and medium companies
http://www.erp5.org