Velocity Reviews > numpy (matrix solver) - python vs. matlab

# numpy (matrix solver) - python vs. matlab

someone
Guest
Posts: n/a

 04-29-2012
Hi,

Notice cross-post, I hope you bear over with me for doing that (and I
imagine that some of you also like python in the matlab-group like
myself)...

------------------------------------------
Python vs. Matlab:
------------------------------------------

Python:
========
from numpy import matrix
from numpy import linalg
A = matrix( [[1,2,3],[11,12,13],[21,22,23]] )
print "A="
print A
print "A.I (inverse of A)="
print A.I

A.I (inverse of A)=
[[ 2.81466387e+14 -5.62932774e+14 2.81466387e+14]
[ -5.62932774e+14 1.12586555e+15 -5.62932774e+14]
[ 2.81466387e+14 -5.62932774e+14 2.81466387e+14]]

Matlab:
========
>> A=[1 2 3; 11 12 13; 21 22 23]

A =

1 2 3
11 12 13
21 22 23

>> inv(A)

Warning: Matrix is close to singular or badly scaled.
Results may be inaccurate. RCOND = 1.067522e-17.

ans =

1.0e+15 *

0.3002 -0.6005 0.3002
-0.6005 1.2010 -0.6005
0.3002 -0.6005 0.3002

------------------------------------------
Python vs. Matlab:
------------------------------------------

So Matlab at least warns about "Matrix is close to singular or badly
scaled", which python (and I guess most other languages) does not...

Which is the most accurate/best, even for such a bad matrix? Is it
possible to say something about that? Looks like python has a lot more
digits but maybe that's just a random result... I mean.... Element 1,1 =
2.81e14 in Python, but something like 3e14 in Matlab and so forth -
there's a small difference in the results...

With python, I would also kindly ask about how to avoid this problem in
the future, I mean, this maybe means that I have to check the condition
number at all times before doing anything at all ? How to do that?

I hope you matlabticians like this topic, at least I myself find it
interesting and many of you probably also program in some other language

someone
Guest
Posts: n/a

 04-30-2012
On 04/30/2012 12:39 AM, Kiuhnm wrote:

>> So Matlab at least warns about "Matrix is close to singular or badly
>> scaled", which python (and I guess most other languages) does not...

>
> A is not just close to singular: it's singular!

Ok. When do you define it to be singular, btw?

>> Which is the most accurate/best, even for such a bad matrix? Is it
>> possible to say something about that? Looks like python has a lot more
>> digits but maybe that's just a random result... I mean.... Element 1,1 =
>> 2.81e14 in Python, but something like 3e14 in Matlab and so forth -
>> there's a small difference in the results...

>
> Both results are *wrong*: no inverse exists.

What's the best solution of the two wrong ones? Best least-squares
solution or whatever?

>> With python, I would also kindly ask about how to avoid this problem in
>> the future, I mean, this maybe means that I have to check the condition
>> number at all times before doing anything at all ? How to do that?

>
> If cond(A) is high, you're trying to solve your problem the wrong way.

So you're saying that in another language (python) I should check the
condition number, before solving anything?

> You should try to avoid matrix inversion altogether if that's the case.
> For instance you shouldn't invert a matrix just to solve a linear system.

What then?

Cramer's rule?

Nasser M. Abbasi
Guest
Posts: n/a

 04-30-2012
On 04/29/2012 05:17 PM, someone wrote:

> I would also kindly ask about how to avoid this problem in
> the future, I mean, this maybe means that I have to check the condition
> number at all times before doing anything at all ? How to do that?
>

I hope you'll check the condition number all the time.

You could be designing a building where people will live in it.

If do not check the condition number, you'll end up with a building that
will fall down when a small wind hits it and many people will die all
because you did not bother to check the condition number when you solved
the equations you used in your design.

Also, as was said, do not use INV(A) directly to solve equations.

--Nasser

Paul Rubin
Guest
Posts: n/a

 04-30-2012
someone <(E-Mail Removed)> writes:
>> A is not just close to singular: it's singular!

> Ok. When do you define it to be singular, btw?

Singular means the determinant is zero, i.e. the rows or columns
are not linearly independent. Let's give names to the three rows:

a = [1 2 3]; b = [11 12 13]; c = [21 22 23].

Then notice that c = 2*b - a. So c is linearly dependent on a and b.
Geometrically this means the three vectors are in the same plane,
so the matrix doesn't have an inverse.

>>> Which is the most accurate/best, even for such a bad matrix?

What are you trying to do? If you are trying to calculate stuff
with matrices, you really should know some basic linear algebra.

someone
Guest
Posts: n/a

 04-30-2012
On 04/30/2012 02:38 AM, Nasser M. Abbasi wrote:
> On 04/29/2012 05:17 PM, someone wrote:
>
>> I would also kindly ask about how to avoid this problem in
>> the future, I mean, this maybe means that I have to check the condition
>> number at all times before doing anything at all ? How to do that?
>>

>
> I hope you'll check the condition number all the time.

So how big can it (cond-number) be before I should do something else?
And what to do then? Cramers rule or pseudoinverse or something else?

> You could be designing a building where people will live in it.
>
> If do not check the condition number, you'll end up with a building that
> will fall down when a small wind hits it and many people will die all
> because you did not bother to check the condition number when you solved
> the equations you used in your design.
>
> Also, as was said, do not use INV(A) directly to solve equations.

In Matlab I used x=A\b.

I used inv(A) in python. Should I use some kind of pseudo-inverse or
what do you suggest?

Nasser M. Abbasi
Guest
Posts: n/a

 04-30-2012
On 04/29/2012 07:59 PM, someone wrote:

>>
>> Also, as was said, do not use INV(A) directly to solve equations.

>
> In Matlab I used x=A\b.
>

good.

> I used inv(A) in python. Should I use some kind of pseudo-inverse or
> what do you suggest?
>

I do not use python much myself, but a quick google showed that pyhton
scipy has API for linalg, so use, which is from the documentation, the
following code example

X = scipy.linalg.solve(A, B)

But you still need to check the cond(). If it is too large, not good.
How large and all that, depends on the problem itself. But the rule of
thumb, the lower the better. Less than 100 can be good in general, but I
really can't give you a fixed number to use, as I am not an expert in
this subjects, others who know more about it might have better
recommendations.

--Nasser

Nasser M. Abbasi
Guest
Posts: n/a

 04-30-2012
On 04/29/2012 07:17 PM, someone wrote:

> Ok. When do you define it to be singular, btw?
>

There are things you can see right away about a matrix A being singular
without doing any computation. By just looking at it.

For example, If you see a column (or row) being a linear combination of
other column(s) (or row(s)) then this is a no no.

1 2 3
11 12 13
21 22 23

You can see right away that if you multiply the second row by 2, and
subtract from that one times the first row, then you obtain the third row.

Hence the third row is a linear combination of the first row and the
second row. no good.

When you get a row (or a column) being a linear combination of others
rows (or columns), then this means the matrix is singular.

--Nasser

Russ P.
Guest
Posts: n/a

 05-01-2012
On Apr 29, 5:17*pm, someone <(E-Mail Removed)> wrote:
> On 04/30/2012 12:39 AM, Kiuhnm wrote:
>
> >> So Matlab at least warns about "Matrix is close to singular or badly
> >> scaled", which python (and I guess most other languages) does not...

>
> > A is not just close to singular: it's singular!

>
> Ok. When do you define it to be singular, btw?
>
> >> Which is the most accurate/best, even for such a bad matrix? Is it
> >> possible to say something about that? Looks like python has a lot more
> >> digits but maybe that's just a random result... I mean.... Element 1,1=
> >> 2.81e14 in Python, but something like 3e14 in Matlab and so forth -
> >> there's a small difference in the results...

>
> > Both results are *wrong*: no inverse exists.

>
> What's the best solution of the two wrong ones? Best least-squares
> solution or whatever?
>
> >> With python, I would also kindly ask about how to avoid this problem in
> >> the future, I mean, this maybe means that I have to check the condition
> >> number at all times before doing anything at all ? How to do that?

>
> > If cond(A) is high, you're trying to solve your problem the wrong way.

>
> So you're saying that in another language (python) I should check the
> condition number, before solving anything?
>
> > You should try to avoid matrix inversion altogether if that's the case.
> > For instance you shouldn't invert a matrix just to solve a linear system.

>
> What then?
>
> Cramer's rule?

If you really want to know just about everything there is to know
about a matrix, take a look at its Singular Value Decomposition (SVD).
I've never used numpy, but I assume it can compute an SVD.

Eelco
Guest
Posts: n/a

 05-01-2012
There is linalg.pinv, which computes a pseudoinverse based on SVD that
works on all matrices, regardless of the rank of the matrix. It merely
approximates A*A.I = I as well as A permits though, rather than being
a true inverse, which may not exist.

Anyway, there are no general answers for this kind of thing. In all
non-textbook problems I can think of, the properties of your matrix
are highly constrained by the problem you are working on; which
additional tests are required to check for corner cases thus depends
on the problem. Often, if you have found an elegant solution to your
problem, no such corner cases exist. In that case, MATLAB is just
wasting your time with its automated checks.

someone
Guest
Posts: n/a

 05-01-2012
On 04/30/2012 02:57 AM, Paul Rubin wrote:
> someone<(E-Mail Removed)> writes:
>>> A is not just close to singular: it's singular!

>> Ok. When do you define it to be singular, btw?

>
> Singular means the determinant is zero, i.e. the rows or columns
> are not linearly independent. Let's give names to the three rows:
>
> a = [1 2 3]; b = [11 12 13]; c = [21 22 23].
>
> Then notice that c = 2*b - a. So c is linearly dependent on a and b.
> Geometrically this means the three vectors are in the same plane,
> so the matrix doesn't have an inverse.

Oh, thak you very much for a good explanation.

>>>> Which is the most accurate/best, even for such a bad matrix?

>
> What are you trying to do? If you are trying to calculate stuff
> with matrices, you really should know some basic linear algebra.

Actually I know some... I just didn't think so much about, before
writing the question this as I should, I know theres also something like
singular value decomposition that I think can help solve otherwise
illposed problems, although I'm not an expert like others in this forum,
I know for sure