Question

What are the advantages of NumPy over regular Python lists?

I have approximately 100 financial markets series, and I am going to create a cube array of 100x100x100 = 1 million cells. I will be regressing (3-variable) each x with each y and z, to fill the array with standard errors.

I have heard that for "large matrices" I should use NumPy as opposed to Python lists, for performance and scalability reasons. Thing is, I know Python lists and they seem to work for me.

What will the benefits be if I move to NumPy?

What if I had 1000 series (that is, 1 billion floating point cells in the cube)?

Answer 1

The advantage of NumPy over regular Python lists are as follows:-

The first thing the NumPy is more efficient as compared to List. It is also more convenient to use. When you use NumPy you get a lot of vector and matrix operations for free, which sometimes allow one to avoid unnecessary work.

An example that illustrates the advantage of NumPy over List, you could read your cube directly from a file into an array by using the following NumPy code:

x = numpy.fromfile(file=open("data"), dtype=float).reshape((100, 100, 100))

These are some useful stuff you can do with the above code:- 

You can sum along the second dimension using the following code:

s = x.sum(axis=1)

You can also find which cells are above a threshold:

(x > 0.5).nonzero()

Also, there are many useful libraries work with NumPy arrays. Some of them are, statistical analysis and visualization libraries.

All these things make NumPy more efficient that becomes the advantage.