Classification using neural networks

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Classification using neural networks

Nuttall, Brandon C

Folks,

 

I am looking for a method to do rock type identification using geophysical log data acquired from oil and gas wells. The geophysical log data are continuous recordings of a variety of bulk rock properties that change with the type and abundance of mineral constituents. What I want to do is classify the rock types (limestone, sandstone, shale, dolomite, etc) and their relative percentages. I remember reading an earlier SCIPY post about a neural network module that used training datasets to establish a classification scheme and then was run against unknown data. But, I can’t find the post and don’t remember the module name. Can anyone point me in the right direction?

 

Thanks.

 

Brandon Nuttall, KRPG-1364

Kentucky Geological Survey

www.uky.edu/kgs

[hidden email] (KGS, Mo-We)

[hidden email] (EEC,Th-Fr)

859-323-0544

859-684-7473 (cell)

 


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

Jonathan T. Niehof
In reply to this post by Nuttall, Brandon C
On 07/25/2012 12:47 PM, Nuttall, Brandon C wrote:

> I am looking for a method to do rock type identification using
> geophysical log data acquired from oil and gas wells. The geophysical
> log data are continuous recordings of a variety of bulk rock properties
> that change with the type and abundance of mineral constituents. What I
> want to do is classify the rock types (limestone, sandstone, shale,
> dolomite, etc) and their relative percentages. I remember reading an
> earlier SCIPY post about a neural network module that used training
> datasets to establish a classification scheme and then was run against
> unknown data. But, I can’t find the post and don’t remember the module
> name. Can anyone point me in the right direction?

Don't know about classification schemes in particular, but there's ffnet
for neural networks:
http://ffnet.sourceforge.net/

--
Jonathan Niehof
ISR-3 Space Data Systems
Los Alamos National Laboratory
MS-D466
Los Alamos, NM 87545

Phone: 505-667-9595
email: [hidden email]

Correspondence /
Technical data or Software Publicly Available
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
In reply to this post by Nuttall, Brandon C
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

Zachary Pincus-2
In reply to this post by Nuttall, Brandon C
Hi Brandon,

Last time I was current with machine learning (ca. 5-7 years ago), the standard advice for the first pass at any particular problem was "throw it at an SVM". I don't know if that's still the go-to consensus these days -- can anyone else weight in? (Does some kind of ensemble method routinely beat SVMs these days in the same way that SVMs were routinely beating neural networks in the early 2000s? I guess I should check out abstracts from recent NIPS conferences to find out...)

Anyway, my suggestion would be to use libSVM (which has handy command-line programs, or python bindings), either from the authors:
http://www.csie.ntu.edu.tw/~cjlin/libsvm/
or as part of the scikits-learn package (which has other classification algorithms rolled in too):
http://scikit-learn.org/stable/

Here is "A practical guide to support vector classification" by the authors of libSVM, which provides best practices for trying to train a SVM from new data:
http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf

Also, libSVM supports multi-class classification (via a one-against-rest strategy IIRC), which looks like it's what you'll need. Though I note that determining percentages of rock types from continuous input data is more of a mixture modeling task, rather than classification (or even regression). Maybe you'd want to look into fitting gaussian (or non-gaussian) mixture models. For this, scikits-learn has various tools also.

Zach





On Jul 25, 2012, at 2:47 PM, Nuttall, Brandon C wrote:

> Folks,
>  
> I am looking for a method to do rock type identification using geophysical log data acquired from oil and gas wells. The geophysical log data are continuous recordings of a variety of bulk rock properties that change with the type and abundance of mineral constituents. What I want to do is classify the rock types (limestone, sandstone, shale, dolomite, etc) and their relative percentages. I remember reading an earlier SCIPY post about a neural network module that used training datasets to establish a classification scheme and then was run against unknown data. But, I can’t find the post and don’t remember the module name. Can anyone point me in the right direction?
>  
> Thanks.
>  
> Brandon Nuttall, KRPG-1364
> Kentucky Geological Survey
> www.uky.edu/kgs
> [hidden email] (KGS, Mo-We)
> [hidden email] (EEC,Th-Fr)
> 859-323-0544
> 859-684-7473 (cell)
>  
> _______________________________________________
> SciPy-User mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/scipy-user

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
In reply to this post by Nuttall, Brandon C
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

Andrew Giessel
scikit-learn is probably a great place to start:  http://scikit-learn.org/stable/

Many algorithms for classification, including NNs.

ag

2012/7/25 <[hidden email]>
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user



--
Andrew Giessel, PhD

Department of Neurobiology, Harvard Medical School
220 Longwood Ave Boston, MA 02115
ph: 617.432.7971 email: [hidden email]

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
In reply to this post by Nuttall, Brandon C
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

Gael Varoquaux
In reply to this post by Andrew Giessel
On Wed, Jul 25, 2012 at 03:28:01PM -0400, Andrew Giessel wrote:
>    scikit-learn is probably a great place to start:
>    http://scikit-learn.org/stable/
>    Many algorithms for classification, including NNs.

Well, it has a perceptron implementation:
http://scikit-learn.org/dev/modules/generated/sklearn.linear_model.Perceptron.html
but not any multilayer-perceptron[*]. Thus, I don't really think that we can
claim that we have neural-network. That said, they are so 1990's :)

Gael

[*] A GSOC was suppose to implement this, this year, but the student
finally went for an internship at Google.
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
In reply to this post by Nuttall, Brandon C
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

Gael Varoquaux
In reply to this post by Zachary Pincus-2
On Wed, Jul 25, 2012 at 03:17:23PM -0400, Zachary Pincus wrote:
> Last time I was current with machine learning (ca. 5-7 years ago), the standard advice for the first pass at any particular problem was "throw it at an SVM".

I think that this is still a good advice. I would say: standardise your
data (for each feature: mean = 0, norm = 1) and throw it into an SVM.

> I don't know if that's still the go-to consensus these days -- can anyone else weight in? (Does some kind of ensemble method routinely beat SVMs these days in the same way that SVMs were routinely beating neural networks in the early 2000s?

If you have heaps of data, you can try random forests or gradient boosted
trees, that work very well.

The scikit-learn has good implementation of all these algorithms, but not
neural-network: they are too old fashion for hip coders to contribute
them ;)

> I guess I should check out abstracts from recent NIPS conferences to find out...)

Nah, NIPS is about mathturbation, not things that work on real data,
these days.

Gael
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
In reply to this post by Nuttall, Brandon C
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

Sturla Molden-2
In reply to this post by Gael Varoquaux
Den 26.07.2012 09:04, skrev Gael Varoquaux:
>
> Well, it has a perceptron implementation:
> http://scikit-learn.org/dev/modules/generated/sklearn.linear_model.Perceptron.html
> but not any multilayer-perceptron[*]. Thus, I don't really think that we can
> claim that we have neural-network. That said, they are so 1990's :)
>

Yeah, it seems that SVMs are more fashionable than ANNs these days. I
don't know why that is, SVMs are slow to train and use, and I have yet
to see that they out-perform an ANN. Perhaps it's because the latest
edition of Numerical Receipes favour them over AANs, because SVMs
supposedly are more transparent and easier to understand (I beg to
differ). Multilayer ANNs trained with Levenberg-Marquardt and error
backpropagation are among the most powerful non-linear regression and
classification tools there are. And by the way, SciPy already has an  
LM-engine to train one (scipy.optimize.leastsq), all it takes is the
code to compute the Jacobian by backpropagation.

Sturla


_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
In reply to this post by Nuttall, Brandon C
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

David Cournapeau
In reply to this post by Sturla Molden-2
On Thu, Jul 26, 2012 at 5:26 PM, Sturla Molden <[hidden email]> wrote:

> Den 26.07.2012 09:04, skrev Gael Varoquaux:
>>
>> Well, it has a perceptron implementation:
>> http://scikit-learn.org/dev/modules/generated/sklearn.linear_model.Perceptron.html
>> but not any multilayer-perceptron[*]. Thus, I don't really think that we can
>> claim that we have neural-network. That said, they are so 1990's :)
>>
>
> Yeah, it seems that SVMs are more fashionable than ANNs these days. I
> don't know why that is, SVMs are slow to train and use, and I have yet
> to see that they out-perform an ANN. Perhaps it's because the latest
> edition of Numerical Receipes favour them over AANs, because SVMs
> supposedly are more transparent and easier to understand (I beg to
> differ). Multilayer ANNs trained with Levenberg-Marquardt and error
> backpropagation are among the most powerful non-linear regression and
> classification tools there are. And by the way, SciPy already has an
> LM-engine to train one (scipy.optimize.leastsq), all it takes is the
> code to compute the Jacobian by backpropagation.

I find Vapnik work on structured risk minimization to be one of the
crown jewel of machine learning (or statistics for that matter), and
would like to believe it is one of the reason why it is/was populat.
ANN also got a bad press because of the history - mentioning neural
network in your publication was a almost-sure way to get your paper
considered badly a couple of years ago I think.

The focus on one technique in particular is fundamentally wrong, I
think (no free lunch and all that). It all depends on your data and
what you're doing, the "use technique X" that sees X changed every few
years is closer to pop culture than science IMO.

David
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
In reply to this post by Nuttall, Brandon C
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

Gael Varoquaux
In reply to this post by Sturla Molden-2
On Thu, Jul 26, 2012 at 06:26:55PM +0200, Sturla Molden wrote:
> I don't know why that is, SVMs are slow to train and use,

Hum. I guess it depends in which settings, but I disagree: well
implemented they are very fast in many settings.

> and I have yet to see that they out-perform an ANN.

They are just way easier to set up and tune. Seting up and tuning an ANN
can be a bit black magic.

> Perhaps it's because the latest edition of Numerical Receipes favour
> them over AANs,

In the machine learning community (which I know well) this is certainly
not an argument. These people are not afraid of implementing complex
algorithms.

> because SVMs supposedly are more transparent and easier to understand
> (I beg to differ).

I actually agree with NRs here.

> Multilayer ANNs trained with Levenberg-Marquardt and error
> backpropagation are among the most powerful non-linear regression and
> classification tools there are.

Granted for non-linearity, but most high-dimensional problem are well
solved with linear model.

> And by the way, SciPy already has an  LM-engine to train one
> (scipy.optimize.leastsq), all it takes is the code to compute the
> Jacobian by backpropagation.

Yeah, well, that's a bit of work, isn't it :)

Gael
_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
In reply to this post by Nuttall, Brandon C
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

Sturla Molden-2
In reply to this post by David Cournapeau
Den 26.07.2012 18:38, skrev David Cournapeau:
> I find Vapnik work on structured risk minimization to be one of the
> crown jewel of machine learning (or statistics for that matter), and
> would like to believe it is one of the reason why it is/was populat.
> ANN also got a bad press because of the history - mentioning neural
> network in your publication was a almost-sure way to get your paper
> considered badly a couple of years ago I think.

They got hyped by neuroscientists who thought more in terms of
"artificial brain" than statistics. In reality, multilayer perceptrons
are just a generalization of linear models with logistic or gaussian
link function.


> The focus on one technique in particular is fundamentally wrong, I
> think (no free lunch and all that). It all depends on your data and
> what you're doing, the "use technique X" that sees X changed every few
> years is closer to pop culture than science IMO.

As with any statistical tool, blind application is never a good idea.

Sturla






_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
Reply | Threaded
Open this post in threaded view
|

Re: Classification using neural networks

L.Ulferts
In reply to this post by Nuttall, Brandon C
Ich befinde mich im Urlaub.
Ihre Nachricht wird nicht weitergereicht,
sondern ab dem 14. August von mir persönlich
bearbeitet werden. In dringenden Fällen können
Sie sich an meine Kollegen Herrn Günterberg und
Roetmann oder meinen Fachvorgesetzten Herrn Taeger
wenden.

Lothar Ulferts

_______________________________________________
SciPy-User mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/scipy-user
12