[SciPy-User] Installing scipy on MacOS X
Locked
12
12
[SciPy-User] Installing scipy on MacOS X
 
|
I'm working towards developing for scipy (staying away from compiled code) and I'm being tripped up by an inability to install scipy (16.0) in a virtualenv on MacOS X. I've tried different variants of python (MacPorts, Homebrew, Official distribution) (2.7.10, 3.4), and have tried multiple fortran compilers (starting with form the recommended site, then from elsewhere). I get the same error message regardless.
ld: symbol(s) not found for architecture x86_64 collect2: error: ld returned 1 exit status error: Command "/usr/local/bin/gfortran -Wall -g -L/usr/local/lib build/temp.macosx-10.6-intel-3.4/scipy/cluster/_vq.o -L/opt/local/lib -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin14/5.1.0 -Lbuild/temp.macosx-10.6-intel-3.4 -ltatlas -ltatlas -ltatlas -lgfortran -o build/lib.macosx-10.6-intel-3.4/scipy/cluster/_vq.so" failed with exit status 1 This does remind me why I left Fortran years ago, but isn't entertaining in the least. Can anyone walk me through a successful process? I'd greatly appreciate the help. FWIW: I do have it "working" within MacPorts, but that doesn't suit my expectation of development. I'd like to be able to merge my code into scipy and submit a pull request, eventually. Best Regards, Joe _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
|
|
On Mon, Jun 15, 2015 at 9:59 PM, Slater Joseph C , PhD, PE <[hidden email]> wrote: I'm working towards developing for scipy (staying away from compiled code) and I'm being tripped up by an inability to install scipy (16.0) in a virtualenv on MacOS X. I've tried different variants of python (MacPorts, Homebrew, Official distribution) (2.7.10, 3.4), and have tried multiple fortran compilers (starting with form the recommended site, then from elsewhere). I get the same error message regardless. This is trying to use ATLAS, do you have that installed? I'd expect all those source builds to use Accelerate. Can you: - give us the full build command you used - put the complete build log in a gist and link to it - check that ``python -c "import numpy; numpy.test('full')`` passes I've been using Homebrew and gfortran 4.9.1 for a year or so without issues. Ralf
_______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
On Mon, Jun 15, 2015 at 1:07 PM, Ralf Gommers <[hidden email]> wrote:
> > > On Mon, Jun 15, 2015 at 9:59 PM, Slater Joseph C , PhD, PE > <[hidden email]> wrote: >> >> I'm working towards developing for scipy (staying away from compiled code) >> and I'm being tripped up by an inability to install scipy (16.0) in a >> virtualenv on MacOS X. I've tried different variants of python (MacPorts, >> Homebrew, Official distribution) (2.7.10, 3.4), and have tried multiple >> fortran compilers (starting with form the recommended site, then from >> elsewhere). I get the same error message regardless. >> >> ld: symbol(s) not found for architecture x86_64 >> collect2: error: ld returned 1 exit status >> error: Command "/usr/local/bin/gfortran -Wall -g -L/usr/local/lib >> build/temp.macosx-10.6-intel-3.4/scipy/cluster/_vq.o -L/opt/local/lib >> -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin14/5.1.0 >> -Lbuild/temp.macosx-10.6-intel-3.4 -ltatlas -ltatlas -ltatlas -lgfortran -o >> build/lib.macosx-10.6-intel-3.4/scipy/cluster/_vq.so" failed with exit >> status 1 > > > This is trying to use ATLAS, do you have that installed? I'd expect all > those source builds to use Accelerate. > > Can you: > - give us the full build command you used > - put the complete build log in a gist and link to it > - check that ``python -c "import numpy; numpy.test('full')`` passes > > I've been using Homebrew and gfortran 4.9.1 for a year or so without issues. There should always be a working from-scratch build procedure in https://github.com/MacPython/scipy-wheels/blob/master/.travis.yml It's in .travis.yml format, and it does some post-build processing that is irrelevant for your case, but at least it is a build that works on a default OSX system. Cheers, Matthew _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
In reply to this post by ralfgommers
> On Jun 15, 2015, at 4:07 PM, Ralf Gommers <[hidden email]> wrote: > <snip> >> </snip> > > This is trying to use ATLAS, do you have that installed? I'd expect all those source builds to use Accelerate. I did not install it myself. My understanding from reading the docs is that it would build without is (using Accelerate). Do I have to do something to force it to recognize/use Accelerate? Likely I do, but shouldn't have to from what I understand. > > Can you: > - give us the full build command you used python setup.py install > - put the complete build log in a gist and link to it First time I've seen gist. Cool! https://gist.github.com/josephcslater/0930741203fd02dbcc0f#file-scipy-install-log-txt > - check that ``python -c "import numpy; numpy.test('full')`` passes Some failures. Log in the https://gist.github.com/josephcslater/0930741203fd02dbcc0f#file-gistfile1-txt numpy was installed in the virtualenv using pip. sudo -H pip3 install numpy matplotlib no errors resulted from that. > > I've been using Homebrew and gfortran 4.9.1 for a year or so without issues. I'm now at 5.10 . I've been interchanging variables, blindly (due to lack of knowledge). Thanks, Joe > > Ralf > <snip> _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
In reply to this post by Matthew Brett
> On Jun 15, 2015, at 4:11 PM, Matthew Brett <[hidden email]> wrote: > > On Mon, Jun 15, 2015 at 1:07 PM, Ralf Gommers <[hidden email]> wrote: >> >> >> On Mon, Jun 15, 2015 at 9:59 PM, Slater Joseph C , PhD, PE >> <[hidden email]> wrote: >>> > > There should always be a working from-scratch build procedure in > https://github.com/MacPython/scipy-wheels/blob/master/.travis.yml > > It's in .travis.yml format, and it does some post-build processing > that is irrelevant for your case, but at least it is a build that > works on a default OSX system. > Well, I was able to create a fork (my first), add the .travis.yml file, mod, push, and see that it all built on travis-ci.org, which seems pretty magical, but for the fact that I have no clue how to build on my own system using the .travis.yml file. Is there a doc on how to use it locally? Joe _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
Hi,
On Mon, Jun 15, 2015 at 6:05 PM, Slater Joseph C , PhD, PE <[hidden email]> wrote: > >> On Jun 15, 2015, at 4:11 PM, Matthew Brett <[hidden email]> wrote: >> >> On Mon, Jun 15, 2015 at 1:07 PM, Ralf Gommers <[hidden email]> wrote: >>> >>> >>> On Mon, Jun 15, 2015 at 9:59 PM, Slater Joseph C , PhD, PE >>> <[hidden email]> wrote: >>>> >> >> There should always be a working from-scratch build procedure in >> https://github.com/MacPython/scipy-wheels/blob/master/.travis.yml >> >> It's in .travis.yml format, and it does some post-build processing >> that is irrelevant for your case, but at least it is a build that >> works on a default OSX system. >> > > Well, I was able to create a fork (my first), add the .travis.yml file, mod, push, and see that it all built on travis-ci.org, which seems pretty magical, but for the fact that I have no clue how to build on my own system using the .travis.yml file. Is there a doc on how to use it locally? Ah - no - I only pointed that out in case it was obvious how to read the recipe there. Here's the relevant bits: """ - sudo installer -pkg archives/gfortran-4.2.3.pkg -target / - pip install cython - pip install numpy==$NUMPY_VERSION - cd scipy - export CC=clang - export CXX=clang++ """ which mean, respectively: * Install gfortran 4.2.3 dual arch build from pkg installer (in fact archived at https://github.com/MacPython/scipy-wheels/blob/master/archives/gfortran-4.2.3.pkg for posterity and this repo); * pip install cython numpy * cd scipy * export CC=clang * export CXX=clang++ * python setup.py install # or whatever local install method you like The repo just assures us that that this still works, and builds our wheels for us when we release... Cheers, Matthew _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
|
|
In reply to this post by Slater Joseph C , PhD, PE
On Mon, Jun 15, 2015 at 11:13 PM, Slater Joseph C , PhD, PE <[hidden email]> wrote:
The numpy.f2py tests are failing with the same error as your scipy build, so this is due to an incorrectly installed numpy or the wrong gfortran. Once that works, scipy build probably also works.
Never use sudo to install something, leads to annoying issues later on.
That's not a stable release right? I would use either 4.2.3 (the one Matthew linked to) or if you use Homebrew then the 4.9.1 that it packages is fine as well. Ralf _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
In reply to this post by Slater Joseph C , PhD, PE
"Slater Joseph C , PhD, PE" <[hidden email]> wrote:
> I'm working towards developing for scipy (staying away from compiled > code) and I'm being tripped up by an inability to install scipy (16.0) in > a virtualenv on MacOS X. Well, this is what I do... Install Xcode from AppStore. Install Xcode command line utilities using the secret handshake $ /usr/bin/xcode-select --install Install gfortran 4.9 from the gfortran wiki into /usr/local/gfortran. This also gives us version 4.9 of gcc and g++. https://gcc.gnu.org/wiki/GFortranBinaries#MacOS Edit .bash_profile to contain this: export PATH=/usr/local/gfortran/bin:$PATH export CXX=clang++ # or export CXX=g++ export CC=clang # or export CC=gcc export FC=gfortran If the python binary was built with llvm-gcc-4.2.1 you should also have this: export CFLAGS="-mmacosx-version-min=10.6" export CXXFLAGS="-stdlib=libstdc++ -mmacosx-version-min=10.6" This is to make sure you link the correct CRT. -stdlib=libstdc++ is only required for clang++. You don't need it if you use g++. After activating the venv, install Cython and Nose $ pip install cython $ pip install nose Build and install Numpy against Accelerate. Pull the source from Github. Use a stabile maintenance branch of numpy, not the master. $ python setup.py build_ext $ python setup.py install Run the test suite and make sure NumPy works correctly. Now we have a venv for development of SciPy. Make a backup of it for later use. To build scipy for scipy for development: First fork scipy on Github. Clone your forked repo into a local repo, add it as origin. Add https://github.com/scipy/scipy.git as upstream. Building SciPy against Accelerate is simply: $ python setup.py build_ext --inplace Now you can run the tests without installing SciPy into your venv. If your development branch works correctly, fetch upstream/master, rebase your development branch on upstream/master, build and run tests again, push to origin, open a PR on Github, wait for Travis CI to timeout, then complain and have someone restart Travis ;) Sturla _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
In reply to this post by ralfgommers
So here I'm a bit lost as to how to install numpy to make it work completely. I've had, nominally, two installations I've tried to get this to work with. One is the MacPorts version, which installs numpy "fine". However, the tests fail there as well.
I thought it was necessary in order to install into the site-packages directory.
I only tried the 5.1 that one time. Tried the 4.2.3 in various configurations since (without luck). _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
In reply to this post by Matthew Brett
> On Jun 15, 2015, at 9:13 PM, Matthew Brett <[hidden email]> wrote: > > Hi, > > On Mon, Jun 15, 2015 at 6:05 PM, Slater Joseph C , PhD, PE > <[hidden email]> wrote: > <snip> > The repo just assures us that that this still works, and builds our > wheels for us when we release... > > Cheers, > > Matthew That will be very helpful if I ever get my system to comply. Unfortunately, the build still failed for me. I think I'm running in circles now. A diff between the results of this run and the one I posted yesterday is substantial. I might need to wipe all my python installs clean and start fresh, perhaps? Happy to if it has a chance of working. I don't have anything special that reinstalling wouldn't restore (via macports). I'm not sure I'm using macports for much more than python and python packages anyway. Joe _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
In reply to this post by Sturla Molden-3
> On Jun 16, 2015, at 7:50 AM, Sturla Molden <[hidden email]> wrote: > > "Slater Joseph C , PhD, PE" <[hidden email]> wrote: >> I'm working towards developing for scipy (staying away from compiled >> code) and I'm being tripped up by an inability to install scipy (16.0) in >> a virtualenv on MacOS X. > > Well, this is what I do... > > Install Xcode from AppStore. already had been done > > Install Xcode command line utilities using the secret handshake > $ /usr/bin/xcode-select --install already had been done (required by macports) > > Install gfortran 4.9 from the gfortran wiki into /usr/local/gfortran. This > also gives us version 4.9 of gcc and g++. > https://gcc.gnu.org/wiki/GFortranBinaries#MacOS > > Edit .bash_profile to contain this: > > export PATH=/usr/local/gfortran/bin:$PATH > export CXX=clang++ # or export CXX=g++ > export CC=clang # or export CC=gcc > export FC=gfortran I'm missing this. Will add this and try when I get back. > > If the python binary was built with llvm-gcc-4.2.1 you should also have > this: How do I determine this? Not sure what direction I should go. I can wipe macports. Not sure if I should try to remove the python.org version instead. What would you suggest? > > export CFLAGS="-mmacosx-version-min=10.6" > export CXXFLAGS="-stdlib=libstdc++ -mmacosx-version-min=10.6" > > This is to make sure you link the correct CRT. -stdlib=libstdc++ is only > required for clang++. You don't need it if you use g++. > > After activating the venv, install Cython and Nose > > $ pip install cython > $ pip install nose > > Build and install Numpy against Accelerate. Not sure how to do this. The the Macports version, I've been using their "port install". For the python.org distribution, I'd been installing using pip, which does it's magic without intervention, as I understand. > Pull the source from Github. > Use a stabile maintenance branch of numpy, not the master. > > $ python setup.py build_ext > $ python setup.py install > > Run the test suite and make sure NumPy works correctly. > > Now we have a venv for development of SciPy. Make a backup of it for later > use. > > To build scipy for scipy for development: > > First fork scipy on Github. Clone your forked repo into a local repo, add > it as origin. Add https://github.com/scipy/scipy.git as upstream. > > Building SciPy against Accelerate is simply: > > $ python setup.py build_ext --inplace > > Now you can run the tests without installing SciPy into your venv. > > If your development branch works correctly, fetch upstream/master, rebase > your development branch on upstream/master, build and run tests again, push > to origin, open a PR on Github, wait for Travis CI to timeout, then > complain and have someone restart Travis ;) > > Sturla > > _______________________________________________ > SciPy-User mailing list > [hidden email] > http://mail.scipy.org/mailman/listinfo/scipy-user _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
"Slater Joseph C , PhD, PE" <[hidden email]> wrote:
> How do I determine this? Good question. It usually is. > Not sure what direction I should go. I can wipe macports. Not sure if I > should try to remove the python.org version instead. What would you suggest? I don't use Macports or Homebrew, nor the python.org version. Right now I have Anaconda, I used to have Enthought Canopy. Get one of those or build your own Python into a local folder. >> Build and install Numpy against Accelerate. > Not sure how to do this. > The the Macports version, I've been using their "port install". For the > python.org distribution, I'd been installing using pip, which does it's > magic without intervention, as I understand. You can "pip install numpy" if the wheel is built against Accelerate. Ask Matthew Brett. Otherwise see below: > Pull the source from Github. >> Use a stabile maintenance branch of numpy, not the master. >> >> $ python setup.py build_ext >> $ python setup.py install Sturla _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
Sturla Molden <[hidden email]> wrote:
> You can "pip install numpy" if the wheel is built against Accelerate. Ask > Matthew Brett. As I understand it, if NumPy is built against ATLAS, then SciPy will try to do them same. And then you will get an error when it is not found. I was just wondering if that was the kind of ATLAS related you got, which is why I said build NumPy from source. Sturla _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
In reply to this post by Sturla Molden-3
Hi,
On Tue, Jun 16, 2015 at 9:18 AM, Sturla Molden <[hidden email]> wrote: > "Slater Joseph C , PhD, PE" <[hidden email]> wrote: > >> How do I determine this? > > Good question. It usually is. > >> Not sure what direction I should go. I can wipe macports. Not sure if I >> should try to remove the python.org version instead. What would you suggest? > > I don't use Macports or Homebrew, nor the python.org version. > > Right now I have Anaconda, I used to have Enthought Canopy. Get one of > those or build your own Python into a local folder. > > >>> Build and install Numpy against Accelerate. > >> Not sure how to do this. >> The the Macports version, I've been using their "port install". For the >> python.org distribution, I'd been installing using pip, which does it's >> magic without intervention, as I understand. > > You can "pip install numpy" if the wheel is built against Accelerate. Ask > Matthew Brett. I use homebrew a lot, but actually not for Python. I use Python.org Python. If you are getting stuck try nuking the Python.org Python directory, something like: rm -rf /Library/Frameworks/Python.framework/Versions/2.7 Then reinstall Python.org Python from the download, make sure that this Python and pip are on the path: $ which python /Library/Frameworks/Python.framework/Versions/2.7/bin/python $ which pip /Library/Frameworks/Python.framework/Versions/2.7/bin/pip Then make sure you have the dual-arch gfortran installed (from the link in the thread above) and: pip install numpy # should install the standard Accelerate-linked wheel pip install cython export CC=clang export CXX=clang++ At that point, you should be able to build scipy OK - let us know if not. Cheers, Matthew _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
In reply to this post by Sturla Molden-3
> On Jun 16, 2015, at 12:25 PM, Sturla Molden <[hidden email]> wrote:
> > Sturla Molden <[hidden email]> wrote: > >> You can "pip install numpy" if the wheel is built against Accelerate. Ask >> Matthew Brett. > > As I understand it, if NumPy is built against ATLAS, then SciPy will try to > do them same. And then you will get an error when it is not found. I was > just wondering if that was the kind of ATLAS related you got, which is why > I said build NumPy from source. > > I'll try doing this next. A lot of directions/support, taking me a long time to iterate around them. What python do you have installed? I'm happy to exactly copy what anyone else has that works. > Sturla > > _______________________________________________ > SciPy-User mailing list > [hidden email] > http://mail.scipy.org/mailman/listinfo/scipy-user _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
Hi,
On Tue, Jun 16, 2015 at 6:12 PM, Slater Joseph C , PhD, PE <[hidden email]> wrote: >> On Jun 16, 2015, at 12:25 PM, Sturla Molden <[hidden email]> wrote: >> >> Sturla Molden <[hidden email]> wrote: >> >>> You can "pip install numpy" if the wheel is built against Accelerate. Ask >>> Matthew Brett. >> >> As I understand it, if NumPy is built against ATLAS, then SciPy will try to >> do them same. And then you will get an error when it is not found. I was >> just wondering if that was the kind of ATLAS related you got, which is why >> I said build NumPy from source. >> >> > > I'll try doing this next. A lot of directions/support, taking me a long time to iterate around them. What python do you have installed? I'm happy to exactly copy what anyone else has that works. Did you try with Python.org Python using the instructions in my last email? I think those instructions should work. If not - what error do you get? You might also consider checking which gfortran you have with $ which gfortran and $ gfortran --version Cheers, Matthew _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
In reply to this post by Slater Joseph C , PhD, PE
"Slater Joseph C , PhD, PE" <[hidden email]> wrote:
> I'll try doing this next. A lot of directions/support, taking me a long > time to iterate around them. What python do you have installed? I'm happy > to exactly copy what anyone else has that works. I have Anaconda from continuum.io (Python 3.4). I use gfortran binaries from the gfortran wiki, not the dual architecture build that Matthew Brett recommends. With Anaconda you can create an environment for developing SciPy like so (from the top of my head): $ conda create --mkdir --prefix ~/scipydev-env python=3.4 numpy=1.9 cython nose pyflakes pep8 Sturla _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
On Tue, Jun 16, 2015 at 6:56 PM, Sturla Molden <[hidden email]> wrote: "Slater Joseph C , PhD, PE" <[hidden email]> wrote: which is 64 bit only. I use gfortran binaries from the gfortran wiki, not the dual architecture which will get you around the dual architecture complications... I take it Anaconda doesn't provide gfortran -- too bad, that would be nice. However, there are a couple on binstar -- maybe worth a shot? $ binstar search gfortran Using binstar api site https://api.anaconda.org Run 'binstar show <USER/PACKAGE>' to get more details: Packages: Name | Version | Package Types | Platforms ------------------------- | ------ | --------------- | --------------- OpenMDAO/libgfortran | 4.8.3 | conda | linux-32, osx-64 dfroger/gfortran | 4.8.2 | conda | osx-64 : The GNU Compiler Collection dfroger/libgfortran | 4.8.2 | conda | osx-64 : The GNU Compiler Collection Found 3 packages -Chris
Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [hidden email] _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
On 17/06/15 20:29, Chris Barker wrote:
> I have Anaconda from continuum.io <http://continuum.io> (Python 3.4). > > > which is 64 bit only. > > I use gfortran binaries from the gfortran wiki, not the dual > architecture > build that Matthew Brett recommends. > > > which will get you around the dual architecture complications... Yes. That is good enough for SciPy development and for my personal use. I can understand why Matthew Brett wants a dual architecture gfortran for his official binary wheels. But that is a totally different matter. > I take it Anaconda doesn't provide gfortran -- too bad, that would be nice. They don't, but they should :) The gfortran maintainers does provide binaries: https://gcc.gnu.org/wiki/GFortranBinaries It supports Fortran, C, C++, Objective C and Objective-C++. It has OpenMP and it builds OpenBLAS, NumPy and SciPy without any fuzz. That is all I personally care about. Sturla _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
Re: Installing scipy on MacOS X
|
|
On 17/06/15 21:02, Sturla Molden wrote:
>> which will get you around the dual architecture complications... > > Yes. > > That is good enough for SciPy development and for my personal use. And for publishing on AppStore it would also be a different story. Sturla _______________________________________________ SciPy-User mailing list [hidden email] http://mail.scipy.org/mailman/listinfo/scipy-user |
«
Return to Scipy-User
|
1 view|%1 views
| Free forum by Nabble | Edit this page |