[Erp5-users] Installation through buildout / slapos

Cédric de Saint Martin cedric.dsm at tiolive.com
Wed Feb 15 14:51:47 CET 2012 

Hi,
On 15 févr. 2012, at 14:23, Evert van de Waal wrote:

> Cedric,
> 
> No joy, the build fails when it tries to download & install ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.7.4-10.tar.bz2. This is getting quite annoying.
I fixed the problem.
> 
> May I suggest moving to one of the 'legacy' versions for ImageMagick, so we don't run into trouble every time they update to a newer minor version? Already, the 6.7.5-3 version that Leo mentioned is old news, the current version is 6.7.5-6. It seems that minor versions have a life time of only a few days, and are not stored in the 'legacy' directory. 6.7.5-6 was created last monday, so any buildouts that do not refer to a legacy ImageMagick but that do use version numbers are broken if they are older than last monday. ImageMagick does supply a version-less download: ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick.tar.bz2, which presumably is a softlink to the latest & greatest. However, using the latest & greatest means that you need to do major regression testing every time one of your subsidary packages is updated. Not a good idea for a system like ERP5, please use 'legacy' versions! It seems you have no automated regression tests running, otherwise you would have spotted these problems yourselves. In that case: definitely stick with tried & tested versions, which will be available for a long time!
In fact, we developed an automated cache system for EVERYTHING we use in production, so we just have to care when developing profiles.
It seems that someone forgot to pull the right button when releasing last ERP5 version (I will investigate more). I am working in fixing everything to have new ERP5 stable version working. In the meantime, only development branch works.
We also of course have automated build tests for development version saying that build failed, sorry to not have reacted earlier.

By the way, for your information, we will soon change this tutorial to use real SlapOS master (vifib.net for example) to deploy ERP5. If you want to know more, everything is already ready at http://www.slapos.org/wiki/osoe-Lecture.SlapOS.Extended.

One more thing, and not the least : we just finished binary cache. What does it mean? That you don't have to worry about compilation and download of imagemagick, it is already done on our servers. You just have to ask ERP5 installation on SlapOS, and it will download from our cache server the whole ERP5 installation. So next stable ERP5 release (expected in ~10 days) will have this feature.

> 
> By the way, saw you on the openworldforum; you have some great idea's!
Thank you very much! What interested you in particular?
> 
> Hoping you can quickly fix http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/erp5:/software/erp5/software.cfg, (as well as http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/erp5:/software/erp5/software.cfg),
> 
> Evert
> 
> 
> Op 15-2-2012 10:49 AM, Cédric de Saint Martin schreef:
>> 
>> Hi Evert,
>> Yes, you are right. The correct URL for ERP5 development profile is : 
>> 
>> http://git.erp5.org/gitweb/slapos.git/blob_plain/refs/heads/erp5:/software/erp5/software.cfg
>> 
>> Checked 3 times.
>> 
>> You will need thereafter an Apache frontend, I can provide apache configuration when needed. We are implemented integrated frontend, but it is not yet available.
>> 
>> Regards,
>> Cedric
>> 
>> On 15 févr. 2012, at 10:25, Evert van de Waal wrote:
>> 
>>> Hi Cedric,
>>> 
>>> Isn't that the same source as the one in the tutorial? Or did you recently update the tutorial?
>>> 
>>> Thx,
>>> Evert
>>> 
>>> Op 14-2-2012 5:13 PM, Cédric de Saint Martin schreef:
>>>> 
>>>> Hi,
>>>> I wrote a mail that I suppose was never sent. I wasn't subscribed to this mailing list, so the mail never arrived here.
>>>> I hope it will help you!
>>>> Here it is : 
>>>> 
>>>> 
>>>> In the step 6 of the slapos tutorial, I think it is better to use
>>>> http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/erp5/software.cfg
>>>> as URL. We haven't released the new code solving the problem yet, so this uses latest development version.
>>>> 
>>>> This URL contains latest ImageMagick version. It should work on every machine.
>>>> 
>>>> Best regards,
>>>> Cedric
>>>> 
>>>> On 14 févr. 2012, at 17:09, Evert van de Waal wrote:
>>>> 
>>>>> Hi Leo,
>>>>> 
>>>>> I have found the location of the error: the file 
>>>>> http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/component/imagemagick/buildout.cfg
>>>>> contains the URL to the erroneous tarball (line 30). But having found it, I still don't know how to fix it.
>>>>> 
>>>>> It is referenced by
>>>>> http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/stack/erp5.cfg, which is referenced by
>>>>> http://git.erp5.org/gitweb/slapos.git/blob_plain/HEAD:/software/erp5/software.cfg, which is set as the 'supply' in step 6 of the howto.
>>>>> 
>>>>> Is this an ERP5 error, or a slapos error? In case of the latter, please excuse me for posting off-topic...
>>>>> 
>>>>> Sorry also for using the wrong subject in my previous mail. I clicked a wrong button and subscribed to the maillist digest, but that is now fixed.
>>>>> 
>>>>> Thanks,
>>>>> Evert
>>>>> 
>>>>> Op 14-2-2012 4:11 PM, Evert van de Waal schreef:
>>>>>> 
>>>>>> Hi Leo,
>>>>>> 
>>>>>> Thanks for your reply!
>>>>>> 
>>>>>> After some digging, it seems that hexagonit is a standard recipe used for building things. Probably, some other package is using hexagonit to build something that includes ImageMagick. 
>>>>>> 
>>>>>> I am trying to find out why ImageMagick being downloaded. In the installed packages, I come across references to another version of ImageMagick, version 6.6.6-1, which seems to be part of ghostscript and Perl (?). But nowhere do I find any reference to this particular version, 6.7.3-10, not even after a full grep on all files in /opt/slapos and /opt/slapgrid. Strange, I would expect a reference to the ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.7.3-10.tar.bz2 location somewhere. I was hoping to hard-code the location you found, but that seems difficult.
>>>>>> 
>>>>>> An other approach I tried was to download the tarball manually and put it in the right location for slapgrid to think it already was downloaded. However, I can't find any other tarballs, so I don't know what the right location is for downloaded tarballs. It seems slapgrid cleans up after itself once packages are properly installed.
>>>>>> 
>>>>>> Any hints / pointers?
>>>>>> 
>>>>>> Thanks,
>>>>>> Evert
>>>>>> 
>>>>>> 
>>>>>> Op 11-2-2012 12:00 PM, erp5-users-request at erp5.org schreef:
>>>>>>> 
>>>>>>> Date: Fri, 10 Feb 2012 16:58:23 +0100
>>>>>>> From: Leonardo Rochael Almeida <leonardo at nexedi.com>
>>>>>>> Subject: Re: [Erp5-users] Installation through buildout / slapos
>>>>>>> To: erp5-users at erp5.org
>>>>>>> Cc: C?dric de Saint Martin <cedric.dsm at tiolive.com>
>>>>>>> Message-ID: <4F353E9F.20607 at nexedi.com>
>>>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>>>> 
>>>>>>> Hi Evert,
>>>>>>> 
>>>>>>> in your output, the error happens as buildout (actually, 
>>>>>>> hexagonit.recipe.download, called by hexagonit.recipe.cmmi) tries to 
>>>>>>> download
>>>>>>> 
>>>>>>> ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.7.3-10.tar.bz2
>>>>>>> 
>>>>>>> That file doesn't seem to exist, at least not at that location, which 
>>>>>>> contains only ImageMagick-6.7.5-3.tar.gz.
>>>>>>> 
>>>>>>> However the file specified can be found at:
>>>>>>> 
>>>>>>> ftp://ftp.imagemagick.org/pub/ImageMagick/legacy/ImageMagick-6.7.3-10.tar.gz
>>>>>>> 
>>>>>>> C?dric, what needs to be fixed so that the right ImageMagic is 
>>>>>>> downloaded from the right location?
>>>>>>> 
>>>>>>> Regards,
>>>>>>> 
>>>>>>> Leo
>>>>>>> 
>>>>>>> On 10-02-2012 11:59, Evert van de Waal wrote:
>>>>>>>> > Hi All,
>>>>>>>> >
>>>>>>>> > I am installing ERP5 according the steps in
>>>>>>>> > http://www.erp5.com/user-install.ERP5.With.SlapOS, on a vanilla Kubuntu
>>>>>>>> > 11.04 virtual machine.
>>>>>>>> >
>>>>>>>> > During step 5, discover slapgrid, ERP5 and all dependencies are
>>>>>>>> > compiled. I get an error message when ImageMagick 6.7.3-10 is installed:
>>>>>>>> > Failed to change directory (the full error text is added below). I
>>>>>>>> > searched on google, but got no solutions, except that the same error
>>>>>>>> > also occured during an ERP5 test in June 2011:
>>>>>>>> > http://mail.tiolive.com/pipermail/erp5-report/20110603/075286.html. That
>>>>>>>> > was for an earlier version of ImageMagick, but looks very similar.
>>>>>>>> >
>>>>>>>> > The buildout below also reports a failure to download ImageMagick due to
>>>>>>>> > an 'unreachable network'. However, this error was reported for all
>>>>>>>> > packages that were installed. Buildout seems to try several sources for
>>>>>>>> > packages. It was able to download it from somewhere, build it, but got
>>>>>>>> > stuck during installation of the things it built. Thus I think the
>>>>>>>> > 'unreachable network' error is unrelated to the failed to change
>>>>>>>> > directory error.
>>>>>>>> >
>>>>>>>> > Does anybody know how to solve this problem?
>>>>>>>> >
>>>>>>>> > Thanks,
>>>>>>>> > Evert
>>>>>>>> >
>>>>>>>> > Installing imagemagick.
>>>>>>>> > imagemagick: [ENV] CPPFLAGS =
>>>>>>>> > -I/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/bzip2/include
>>>>>>>> > -I/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/zlib/include
>>>>>>>> > -I/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/jbigkit/include
>>>>>>>> > -I/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/libjpeg/include
>>>>>>>> > -I/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/libtiff/include
>>>>>>>> > -I/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/libpng/include
>>>>>>>> > -I/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/jasper/include
>>>>>>>> > -I/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/freetype/include
>>>>>>>> > imagemagick: [ENV] LDFLAGS =
>>>>>>>> > -L/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/bzip2/lib
>>>>>>>> > -Wl,-rpath=/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/bzip2/lib
>>>>>>>> > -L/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/zlib/lib
>>>>>>>> > -Wl,-rpath=/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/zlib/lib
>>>>>>>> > -L/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/jbigkit/lib
>>>>>>>> > -Wl,-rpath=/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/jbigkit/lib -L/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/libjpeg/lib
>>>>>>>> > -Wl,-rpath=/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/libjpeg/lib -L/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/libtiff/lib
>>>>>>>> > -Wl,-rpath=/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/libtiff/lib -L/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/libpng/lib
>>>>>>>> > -Wl,-rpath=/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/libpng/lib
>>>>>>>> > -L/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/jasper/lib
>>>>>>>> > -Wl,-rpath=/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/jasper/lib
>>>>>>>> > -L/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/freetype/lib
>>>>>>>> > -Wl,-rpath=/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/freetype/lib
>>>>>>>> > imagemagick: [ENV] PATH =
>>>>>>>> > /opt/slapos/afaeaed482578cde211ab84114e0c317/parts/freetype/bin:/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/ghostscript/bin:/opt/slapos/afaeaed482578cde211ab84114e0c317/parts/pkgconfig/bin:/opt/slapos/parts/bison/bin:/opt/slapos/parts/bzip2/bin:/opt/slapos/parts/gettext/bin:/opt/slapos/parts/glib/bin:/opt/slapos/parts/libxml2/bin:/opt/slapos/parts/libxslt/bin:/opt/slapos/parts/m4/bin:/opt/slapos/parts/ncurses/bin:/opt/slapos/parts/openssl/bin:/opt/slapos/parts/pkgconfig/bin:/opt/slapos/parts/python2.7/bin:/opt/slapos/parts/readline/bin:/opt/slapos/parts/sqlite3/bin:/opt/slapos/parts/swig/bin:/opt/slapos/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin
>>>>>>>> >
>>>>>>>> > imagemagick: [ENV] PKG_CONFIG_PATH =
>>>>>>>> > /opt/slapos/afaeaed482578cde211ab84114e0c317/parts/fontconfig/lib/pkgconfig
>>>>>>>> > Downloading
>>>>>>>> > ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.7.3-10.tar.bz2
>>>>>>>> > Downloading ImageMagick-6.7.3-10.tar.bz2 from network cache.
>>>>>>>> > Failed to download from network cache ImageMagick-6.7.3-10.tar.bz2:
>>>>>>>> > <urlopen error [Errno 101] Network is unreachable>
>>>>>>>> > While:
>>>>>>>> > Installing imagemagick.
>>>>>>>> >
>>>>>>>> > An internal error occurred due to a bug in either zc.buildout or in a
>>>>>>>> > recipe being used:
>>>>>>>> > Traceback (most recent call last):
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/eggs/zc.buildout-1.6.0_dev_SlapOS_003-py2.7.egg/zc/buildout/buildout.py",
>>>>>>>> > line 2027, in main
>>>>>>>> > getattr(buildout, command)(args)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/eggs/zc.buildout-1.6.0_dev_SlapOS_003-py2.7.egg/zc/buildout/buildout.py",
>>>>>>>> > line 737, in install
>>>>>>>> > installed_files = self[part]._call(recipe.install)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/eggs/zc.buildout-1.6.0_dev_SlapOS_003-py2.7.egg/zc/buildout/buildout.py",
>>>>>>>> > line 1490, in _call
>>>>>>>> > return f()
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/eggs/hexagonit.recipe.cmmi-1.5.0-py2.7.egg/hexagonit/recipe/cmmi/__init__.py",
>>>>>>>> > line 144, in install
>>>>>>>> > self.buildout, self.name, opt).install()
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/eggs/hexagonit.recipe.download-1.5.0-py2.7.egg/hexagonit/recipe/download/__init__.py",
>>>>>>>> > line 81, in install
>>>>>>>> > path, is_temp = download(self.options['url'],
>>>>>>>> > md5sum=self.options.get('md5sum'))
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/eggs/zc.buildout-1.6.0_dev_SlapOS_003-py2.7.egg/zc/buildout/download.py",
>>>>>>>> > line 119, in __call__
>>>>>>>> > local_path, is_temp = self.download_cached(url, md5sum)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/eggs/zc.buildout-1.6.0_dev_SlapOS_003-py2.7.egg/zc/buildout/download.py",
>>>>>>>> > line 167, in download_cached
>>>>>>>> > _, is_temp = self.download(url, md5sum, cached_path)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/eggs/zc.buildout-1.6.0_dev_SlapOS_003-py2.7.egg/zc/buildout/download.py",
>>>>>>>> > line 209, in download
>>>>>>>> > tmp_path, headers = urllib.urlretrieve(url, tmp_path)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/rebootstrap.3.parts/python2.7/lib/python2.7/urllib.py",
>>>>>>>> > line 91, in urlretrieve
>>>>>>>> > return _urlopener.retrieve(url, filename, reporthook, data)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/rebootstrap.3.parts/python2.7/lib/python2.7/urllib.py",
>>>>>>>> > line 237, in retrieve
>>>>>>>> > fp = self.open(url, data)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/rebootstrap.3.parts/python2.7/lib/python2.7/urllib.py",
>>>>>>>> > line 205, in open
>>>>>>>> > return getattr(self, name)(url)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/rebootstrap.3.parts/python2.7/lib/python2.7/urllib.py",
>>>>>>>> > line 548, in open_ftp
>>>>>>>> > (fp, retrlen) = self.ftpcache[key].retrfile(file, type)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/rebootstrap.3.parts/python2.7/lib/python2.7/urllib.py",
>>>>>>>> > line 898, in retrfile
>>>>>>>> > self.ftp.cwd(file)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/rebootstrap.3.parts/python2.7/lib/python2.7/ftplib.py",
>>>>>>>> > line 543, in cwd
>>>>>>>> > return self.voidcmd(cmd)
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/rebootstrap.3.parts/python2.7/lib/python2.7/ftplib.py",
>>>>>>>> > line 249, in voidcmd
>>>>>>>> > return self.voidresp()
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/rebootstrap.3.parts/python2.7/lib/python2.7/ftplib.py",
>>>>>>>> > line 224, in voidresp
>>>>>>>> > resp = self.getresp()
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/afaeaed482578cde211ab84114e0c317/rebootstrap.3.parts/python2.7/lib/python2.7/ftplib.py",
>>>>>>>> > line 219, in getresp
>>>>>>>> > raise error_perm, resp
>>>>>>>> > IOError: [Errno ftp error] [Errno ftp error] 550 Failed to change
>>>>>>>> > directory.
>>>>>>>> > ERROR:SoftwareReleases:Traceback (most recent call last):
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/eggs/slapos.core-0.19-py2.7.egg/slapos/grid/slapgrid.py",
>>>>>>>> > line 406, in processSoftwareReleaseList
>>>>>>>> > shadir_key_file=self.shadir_key_file).install()
>>>>>>>> > File
>>>>>>>> > "/opt/slapos/eggs/slapos.core-0.19-py2.7.egg/slapos/grid/SlapObject.py",
>>>>>>>> > line 116, in install
>>>>>>>> > console=self.console)
>>>>>>>> > File "/opt/slapos/eggs/slapos.core-0.19-py2.7.egg/slapos/grid/utils.py",
>>>>>>>> > line 318, in launchBuildout
>>>>>>>> > raise BuildoutFailedError(message)
>>>>>>>> > BuildoutFailedError: Failed to run buildout profile in directory
>>>>>>>> > '/opt/slapos/afaeaed482578cde211ab84114e0c317':
>>>>>>>> > Please consult messages above
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > INFO:SoftwareReleases:Finished software releases...
>>>>>>>> > INFO:ComputerPartitionProcessing:Processing computer partitions...
>>>>>>>> > INFO:SVCBackend:Supervisord already running.
>>>>>>>> > INFO:ComputerPartitionProcessing:Finished computer partitions...
>>>>>>>> > INFO:UsageReporting:Aggregating and sending usage reports...
>>>>>>>> > INFO:UsageReporting:Finished usage reports...
>>>>>>>> >
>>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> Erp5-users mailing list
>>>>> Erp5-users at erp5.org
>>>>> https://mail.tiolive.com/mailman/listinfo/erp5-users
>>>> 
>>>> ---------------------
>>>> Cédric de Saint Martin,
>>>> SlapOS Community Manager @ ViFiB
>>>> http://www.slapos.org
>>>> +33 6 81 85 97 18
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Erp5-users mailing list
>>>> Erp5-users at erp5.org
>>>> https://mail.tiolive.com/mailman/listinfo/erp5-users
>>> 
>>> _______________________________________________
>>> Erp5-users mailing list
>>> Erp5-users at erp5.org
>>> https://mail.tiolive.com/mailman/listinfo/erp5-users
>> 
>> ---------------------
>> Cédric de Saint Martin,
>> SlapOS Community Manager @ ViFiB
>> http://www.slapos.org
>> +33 6 81 85 97 18
>> 
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Erp5-users mailing list
>> Erp5-users at erp5.org
>> https://mail.tiolive.com/mailman/listinfo/erp5-users
> 
> _______________________________________________
> Erp5-users mailing list
> Erp5-users at erp5.org
> https://mail.tiolive.com/mailman/listinfo/erp5-users

---------------------
Cédric de Saint Martin,
SlapOS Community Manager @ ViFiB
http://www.slapos.org
+33 6 81 85 97 18




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.tiolive.com/pipermail/erp5-users/attachments/20120215/5c96ffc7/attachment-0001.htm>

More information about the Erp5-users mailing list