Messages (38) msg60482 - (view) Author: Dick Dunbar (dickdunbar) Date: 2004-04-24 15:21

By default, the ld command only looks for *.a archives
to resolve library references.

To recognize dynamic shared libaries (*.so), two flags
need to be added to the Modules/ld_so_aix script: -brtl
-bnortllib

Line 170 in the script should read:

CCOPT="$CCOPT -Wl,-bM:SRE -Wl,-T512 -Wl,-H512 -brtl
-bnortllib -lm -o $objfile"

-brtl ... says you want to include libtk8.4.so 
   when searching for the library -ltk8.4

-bnortllib ... says you do not want the semantics of
the AIX "run time linking" services ... just the search
for the library.

This resolves a problem when Python is built with tk
support.

msg60483 - (view) Author: Neal Norwitz (nnorwitz) * Date: 2005-10-03 03:58

Logged In: YES 
user_id=33168

I added a comment based on the above information.  I don't
want to make a real change, since I can't test it.  If we
get access to some AIX boxes, we could really fix.  I'm
hestitant to break stuff without testing.

Checked in as:

Modules/ld_so_aix 2.6

msg70894 - (view) Author: Sébastien Sablé (sable) Date: 2008-08-08 09:44

Hi,

I have modified a bit this patch to make it work with Python 2.5.2 (see
attached patched).

It is a great enhancement for Python users on AIX to be able to use a
shared library instead of a static one, so I would appreciate if this
feature could be integrated in trunk.

We have various AIX 5.2 and 5.3 servers in the Sungard company where I
work, and we could run some tests according to your instructions. It may
also be possible to provide some limited access to one of those servers
if needed but it would require some time and paper work.

What kind of access would you need in order to validate those patchs?

thanks in advance

msg70896 - (view) Author: Antoine Pitrou (pitrou) * Date: 2008-08-08 10:02

Sébastien, what does your patch change exactly? Does it build a shared
lib for Python by default? If that's the case, I'm not sure it's a good
idea due to AIX's particular way of treating shared libraries; it might
break e.g. when trying to embed Python in another program.
(I say this while being an AIX novice, but I did have to compile Python
under AIX for embedded use in another third-party software...
unfortunately right now I don't have access to an AIX machine with a
compiler)

msg70902 - (view) Author: Sébastien Sablé (sable) Date: 2008-08-08 12:30

Hum, I tried to recompile Python without the --enable-shared option on
AIX and it crashed; so this patch may not be compatible with a static
version of Python (though I have other patches in my version - cf issue
3526 - which may be incompatible).

I will do a bit of cleanup in my environment and try to make a version
of the patch that works also when compiling a static Python.

msg88115 - (view) Author: Loris Bennett (loris) Date: 2009-05-20 13:20

Shouldn't line 170 be

   CCOPT="$CCOPT -Wl,-bM:SRE -Wl,-T512 -Wl,-H512 -Wl,-brtl
-Wl,-bnortllib -lm -o $objfile"

instead of

   CCOPT="$CCOPT -Wl,-bM:SRE -Wl,-T512 -Wl,-H512 -brtl -bnortllib -lm -o
$objfile"

Otherwise the -brtl and -bnortllib flags are interpreted as compiler
flags and errors like the following will occur:

   gcc: unrecognized option '-brtl'

msg88116 - (view) Author: Loris Bennett (loris) Date: 2009-05-20 13:33

Addition to last message:

More importantly, without "-Wl", the flags are not passed to the linker
and the tcl/tk libs will not be found.

msg115262 - (view) Author: Sébastien Sablé (sable) Date: 2010-08-31 13:32

Hi,

As reported in this issue and issue 1756343 and issue 1542544, Python does not produce a shared python library on AIX even with the --enable-shared flag.

I had provided a patch to correct that, but it was breaking static compilation of Python on AIX.

We are upgrading our AIX build environment to AIX 6.1 and Python 2.6.6, so I took the time to review this patch again and to clean it so that it works well when compiling Python statically and dynamically, with either gcc and xlc_r.

I attach the resulting patch, and the build and unit test logs. You can notice that not only this patch allows to compile Python dynamically on AIX, but also that it improves the unit tests results when compiling Python statically.

For example with the xlc_r compiler and Python compiled statically, the results without the patch are:
  274 tests OK.
  22 tests failed
  70 tests skipped
The results with the patch are:
  286 tests OK.
  21 tests failed
  59 tests skipped

A minor issue with this patch is that "libpython2.6.so" is hardcoded in the ld_so_aix file. That should be modified by moving ld_so_aix to ld_so_aix.in and using $LDLIBRARY, or by using an explicit flag when calling ld_so_aix to compile libpythonx.y.so.

If you agree to the principle of this patch, I can make the extra work to clean ld_so_aix and test this patch on Python 2.7 and 3.1.

regards

msg115279 - (view) Author: Antoine Pitrou (pitrou) * Date: 2010-08-31 22:16

In dynamic builds, there seem to be lots of messages such as:

ld: 0711-224 WARNING: Duplicate symbol: .PyLong_FromString
ld: 0711-224 WARNING: Duplicate symbol: PyLong_FromString

Are you sure this is normal?

In any case, if you find a way to cleanup ld_so_aix, this should probably be checked in anyway. I doubt anybody else tests under AIX, so it's reasonable to trust you on this.

Even better if you manage to diagnose and/or fix some of the test failures, by the way. (if not, you can still open separate issues for each of them)

msg115375 - (view) Author: Sébastien Sablé (sable) Date: 2010-09-02 13:58

Le 01/09/2010 00:16, Antoine Pitrou a écrit :
> In dynamic builds, there seem to be lots of messages such as:
>
> ld: 0711-224 WARNING: Duplicate symbol: .PyLong_FromString
> ld: 0711-224 WARNING: Duplicate symbol: PyLong_FromString
>
> Are you sure this is normal?

This is not normal, but I always had this warning and this does not 
prevent the libraries from working OK.
I will try to see if I can clean that.

> In any case, if you find a way to cleanup ld_so_aix, this should probably be checked in anyway. I doubt anybody else tests under AIX, so it's reasonable to trust you on this.
>
> Even better if you manage to diagnose and/or fix some of the test failures, by the way. (if not, you can still open separate issues for each of them)

Great! this is the kind of answer I was looking for. I will cleanup my 
patch to make it compatible with Python 2.7 and Python 3.1.2 and test it 
with those 2 versions.

I will also correct as many unitary tests as possible with this platform.
I already have a patch for issue 4026.

msg115703 - (view) Author: Sébastien Sablé (sable) Date: 2010-09-06 13:57

Hi,

I have updated the patch and adapted it for Python 3.1.2:

* this time ld_so_aix is moved to ld_so_aix.in which is converted to ld_so_aix by 'configure' so that it has the right version number for the python library

* there are no more warnings for 'Duplicate symbol': I changed build_ext.py to not link extensions with libpython since the symbols are already defined by python.exp

* I made a modification to ld_so_aix specifically for Python 3 as according to pep-3121 the entry function has been renamed from init<modulname> to PyInit_<modulename>

I will provide another patch for Python 2.7 (nearly the same, but without the modification needed by pep-3121).
I also attach the build log for Python 3.1.2 without and with this patch.

regards

msg115707 - (view) Author: Antoine Pitrou (pitrou) * Date: 2010-09-06 14:27

Thank you. For the record, here is the patch adapted for 3.2 development branch. It looks ok to me.

msg115708 - (view) Author: Sébastien Sablé (sable) Date: 2010-09-06 14:35

Here is the patch for Python 2.7.

The only difference compared to Python 3.1.2 is that we don't rename init<modulname> to PyInit_<modulename>.

msg115709 - (view) Author: Antoine Pitrou (pitrou) * Date: 2010-09-06 14:59

> * there are no more warnings for 'Duplicate symbol': I changed
> build_ext.py to not link extensions with libpython since the symbols
> are already defined by python.exp

Could this affect building extensions with gcc under AIX? Or does gcc delegate to the AIX linker?

msg115729 - (view) Author: Sébastien Sablé (sable) Date: 2010-09-06 19:22

Yes Antoine, gcc only handles compilation; the linker is explicitly called through the ld_so_aix script which handles calling the native ld with the right flags to import symbols.

I will check with gcc and attach the log on Wednesday (not at work tomorrow) but there should be no problem.

msg115731 - (view) Author: Sébastien Sablé (sable) Date: 2010-09-06 19:40

Hum, forget my previous note;I checked ld_so_aix and it actually calls $CC to handle linking not ld. I suppose gcc will call the native ld.

Anyway, I will run the test with gcc on Wednesday and know for sure if that works.

msg115734 - (view) Author: Antoine Pitrou (pitrou) * Date: 2010-09-06 20:22

> Anyway, I will run the test with gcc on Wednesday and know for sure if
> that works.

Thank you! If it works, it's good for commit.

msg115865 - (view) Author: Sébastien Sablé (sable) Date: 2010-09-08 12:42

Here are the build logs for Python 3.1.2 modified with this patch and compiled with gcc.
The extensions are correctly built.

I also compiled an extension not directly provided in Python source (cx_Oracle) and it worked fine also.

msg115876 - (view) Author: Sébastien Sablé (sable) Date: 2010-09-08 14:26

Antoine, I wanted to test this improvement (and others) on the branch py3k (I was using Python 2.7 and Python 3.1.2 for the moment). But I have some issues compiling this branch, even without any patch (see issue 9799).

So you may want to wait for the other issue to be resolved before to commit this patch on branch py3k.

msg115987 - (view) Author: Sébastien Sablé (sable) Date: 2010-09-10 09:24

Antoine, I tested this patch on py3k with both gcc and xlc in static and shared mode and I did not notice any issue.

I attach the build and test logs.

I think you can safely commit it.

msg116040 - (view) Author: Antoine Pitrou (pitrou) * Date: 2010-09-10 20:07

I've committed the latest patch in r84680 (3.x), r84682 (3.1) and r84683 (2.7). Perhaps you want to check it's ok. Thank you anyway!

msg116042 - (view) Author: Sébastien Sablé (sable) Date: 2010-09-10 21:09

Great! Thanks Antoine.

I checked quickly and there is a small correction to do on the 2.7 branch: this branch is different because there has been a change between python 2.x and 3.x in pep-3121 concerning the name of the entry function in a module (init<modulname> to PyInit_<modulename>)

So you need to change this line in Modules/ld_so_aix.in from:
  entry=PyInit_`echo $filename | sed "s/module.*//"`
to
  entry=init`echo $filename | sed "s/module.*//"`

Except for that it looks fine.

I will continue to try to make all unitary tests pass on AIX. And I would like to setup a buildbot that will compile and test the trunk every day and post the log on a public web site, in order to help keep this platform well supported.

msg128214 - (view) Author: Michael Haubenwallner (haubi) * Date: 2011-02-09 14:49

Sébastien, why did you need to add '-L$(srcdir)' to BLDSHARED in these patches?

As LDLIBRARY points to the immediate file 'libpython$(VERSION).so' instead of '-lpython$(VERSION)', I don't see the need for '-L$(srcdir)' at all.

The problem is that '-L$(srcdir)' adds '$(srcdir)' to the runpath too (as there is no '-blibpath' argument), opening a security hole for libpythonX.Y.so as well as the modules.so.

msg128281 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-10 11:11

Hi Micheal,

I don't remember why "-L\$(srcdir)" was added since it has been quite a long time.

I tried to recompile everything after removing it and everything compiled just fine, so I guess it can be removed.

regards

msg128586 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-15 12:06

Hum, I was incorrect in previous note:
You can remove -L\$(srcdir) on this line, but then you need to specify the full path to Modules/python.exp. See patch below:

Index: configure.in
===================================================================
--- configure.in        (revision 88422)
+++ configure.in        (working copy)
@@ -1642,7 +1655,7 @@
 then
        case $ac_sys_system/$ac_sys_release in
        AIX*)
-               BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:Modules/python.exp -L\$(srcdir)"
+               BLDSHARED="\$(srcdir)/Modules/ld_so_aix \$(CC) -bI:\$(srcdir)/Modules/python.exp"
                LDSHARED="\$(BINLIBDEST)/config/ld_so_aix \$(CC) -bI:\$(BINLIBDEST)/config/python.exp"
                ;;
        IRIX/5*) LDSHARED="ld -shared";;


Also, I think there are other issues with building Python 3.2 with shared libraries. I am currently investigating.

msg128591 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-15 13:00

OK, Python with shared libraries is broken in trunk since the library was renamed to libpython3.2m.
Here is a patch to correct that:

Index: Modules/ld_so_aix.in
===================================================================
--- Modules/ld_so_aix.in        (revision 88422)
+++ Modules/ld_so_aix.in        (working copy)
@@ -131,7 +131,7 @@
   shift
 done
 
-if test "$objfile" = "libpython@VERSION@.so"; then
+if test "$objfile" = "libpython@VERSION@@ABIFLAGS@.so"; then
   ldsocoremode="true" 
 fi

msg128592 - (view) Author: Georg Brandl (georg.brandl) * Date: 2011-02-15 13:12

This looks like it should (and could) go into 3.2 final.  Agreed?

msg128593 - (view) Author: Antoine Pitrou (pitrou) * Date: 2011-02-15 13:13

Agreed.

msg128594 - (view) Author: Georg Brandl (georg.brandl) * Date: 2011-02-15 13:18

OK. Sébastien, could you make and attach a complete patch?

msg128596 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-15 13:47

Here is the full patch.
I also modified Makefile.pre.in to clean Modules/python.exp when doing distclean.

msg128597 - (view) Author: Antoine Pitrou (pitrou) * Date: 2011-02-15 13:53

Looks good to me.

msg128602 - (view) Author: Georg Brandl (georg.brandl) * Date: 2011-02-15 15:44

Committed in r88426.

msg128634 - (view) Author: Michael Haubenwallner (haubi) * Date: 2011-02-16 09:31

> You can remove -L\$(srcdir) on this line, but then you need to specify > the full path to Modules/python.exp. See patch below:

Interesting, didn't experience this to be necessary with Python-2.7.1 here... Maybe because I do an in-source build, haven't tried an out-of-source build at all.

msg128638 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-16 09:57

Are you using --enable-shared when compiling python?
That is when you should see the error (it could not find python.exp at some stage).

msg128639 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-16 09:58

By the way... thanks Georg!

msg128640 - (view) Author: Sébastien Sablé (sable) Date: 2011-02-16 10:01

Also Michael, I am working on the 3.2 branch, it may be different with Python 2.7.1 but I haven't tested yet.

msg128652 - (view) Author: Antoine Pitrou (pitrou) * Date: 2011-02-16 11:04

Georg, the non-abiflags portion seems to need backporting.

msg129355 - (view) Author: Georg Brandl (georg.brandl) * Date: 2011-02-25 11:20

Backported to 3.1 in r88560, 2.7 in r88568.

History Date User Action Args 2022-04-11 14:56:03adminsetgithub: 40186 2011-02-26 08:13:30pitroulinkissue11172 superseder 2011-02-25 11:20:59georg.brandlsetstatus: open -> closed
nosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg129355
2011-02-16 11:04:09pitrousetstatus: closed -> open
priority: release blocker -> normal
nosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128652

assignee: georg.brandl

2011-02-16 10:01:18sablesetnosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128640 2011-02-16 09:58:26sablesetnosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128639 2011-02-16 09:57:23sablesetnosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128638 2011-02-16 09:31:53haubisetnosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128634 2011-02-15 15:44:59georg.brandlsetstatus: open -> closed
nosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128602
2011-02-15 13:53:16pitrousetnosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128597 2011-02-15 13:47:05sablesetfiles: + patch_aix_shared_32.diff
nosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128596
2011-02-15 13:18:06georg.brandlsetnosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128594 2011-02-15 13:13:35pitrousetnosy: barry, nnorwitz, georg.brandl, pitrou, sable, benjamin.peterson, dickdunbar, loris, haubi
messages: + msg128593 2011-02-15 13:12:09georg.brandlsetstatus: closed -> open
priority: normal -> release blocker

nosy: + barry, benjamin.peterson, georg.brandl
messages: + msg128592

2011-02-15 13:00:20sablesetnosy: nnorwitz, pitrou, sable, dickdunbar, loris, haubi
messages: + msg128591 2011-02-15 12:06:31sablesetnosy: nnorwitz, pitrou, sable, dickdunbar, loris, haubi
messages: + msg128586 2011-02-10 11:11:58sablesetnosy: nnorwitz, pitrou, sable, dickdunbar, loris, haubi
messages: + msg128281 2011-02-09 14:49:41haubisetnosy: + haubi
messages: + msg128214
2010-09-24 12:21:15pitroulinkissue1542544 superseder 2010-09-10 21:09:34sablesetmessages: + msg116042 2010-09-10 20:07:10pitrousetstatus: open -> closed
resolution: fixed
messages: + msg116040

stage: patch review -> resolved

2010-09-10 15:40:29sablesetfiles: + py3k_aix61_shared_xlc_pathshared.txt.gz 2010-09-10 15:40:07sablesetfiles: - py3k_aix61_shared_xlc_pathshared.txt.gz 2010-09-10 09:26:07sablesetfiles: + py3k_aix61_shared_gcc_patchshared.txt.gz 2010-09-10 09:25:52sablesetfiles: + py3k_aix61_static_gcc_patchshared.txt.gz 2010-09-10 09:25:29sablesetfiles: + py3k_aix61_shared_xlc_pathshared.txt.gz 2010-09-10 09:25:05sablesetfiles: + py3k_aix61_static_xlc_patchshared.txt.gz 2010-09-10 09:24:39sablesetfiles: + py3k_aix61_static_xlc_nopatch.txt.gz

messages: + msg115987

2010-09-08 14:26:14sablesetmessages: + msg115876 2010-09-08 12:43:13sablesetfiles: + python312_AIX61_shared_gcc_patchshared.txt.gz 2010-09-08 12:42:14sablesetfiles: + python312_AIX61_static_gcc_patchshared.txt.gz

messages: + msg115865

2010-09-06 20:22:30pitrousetmessages: + msg115734 2010-09-06 19:40:48sablesetmessages: + msg115731 2010-09-06 19:22:28sablesetmessages: + msg115729 2010-09-06 14:59:18pitrousetmessages: + msg115709 2010-09-06 14:35:18sablesetfiles: + Python-2.7_shared_AIX.diff

messages: + msg115708
versions: + Python 2.6, - Python 3.2

2010-09-06 14:27:33pitrousetfiles: + aix32.diff

stage: test needed -> patch review
messages: + msg115707
versions: + Python 3.2, - Python 2.6

2010-09-06 13:59:40sablesetversions: + Python 2.6, Python 3.1 2010-09-06 13:59:06sablesetfiles: + python312_AIX61_shared_patch_shared.txt.gz 2010-09-06 13:58:39sablesetfiles: + python312_AIX61_static_patch_shared.txt.gz 2010-09-06 13:58:07sablesetfiles: + python312_AIX61_static_nopatch.txt.gz 2010-09-06 13:57:06sablesetfiles: + Python-3.1.2_shared_AIX.diff

messages: + msg115703

2010-09-02 13:58:28sablesetmessages: + msg115375 2010-08-31 22:16:15pitrousetmessages: + msg115279 2010-08-31 13:38:22sablesetfiles: + python266_AIX61_shared_xlc_patchshared.txt.gz 2010-08-31 13:38:06sablesetfiles: + python266_AIX61_shared_xlc_nopatch.txt.gz 2010-08-31 13:37:47sablesetfiles: + python266_AIX61_shared_gcc_patchshared.txt.gz 2010-08-31 13:37:34sablesetfiles: + python266_AIX61_shared_gcc_nopatch.txt.gz 2010-08-31 13:36:58sablesetfiles: + python266_AIX61_static_xlc_patchshared.txt.gz 2010-08-31 13:36:37sablesetfiles: + python266_AIX61_static_xlc_nopatch.txt.gz 2010-08-31 13:36:07sablesetfiles: + python266_AIX61_static_gcc_patchshared.txt.gz 2010-08-31 13:35:32sablesetfiles: + python266_AIX61_static_gcc_nopatch.txt.gz 2010-08-31 13:32:54sablesetfiles: + Python-2.6.6_shared_AIX.diff

messages: + msg115262

2009-05-20 13:33:00lorissetmessages: + msg88116 2009-05-20 13:20:08lorissetnosy: + loris
messages: + msg88115
2009-02-14 13:54:38ajaksu2setstage: test needed
type: enhancement
components: + Build, Extension Modules, - None
versions: + Python 2.7 2008-08-08 12:30:08sablesetmessages: + msg70902 2008-08-08 10:02:57pitrousetnosy: + pitrou
messages: + msg70896 2008-08-08 09:44:15sablesetfiles: + patch_AIX_shared.diff
keywords: + patch
messages: + msg70894
nosy: + sable 2004-04-24 15:21:59dickdunbarcreate