NumPy

NumPy (Numerical Python) serves as the foundational package for scientific computing and data analysis. It forms the building block for other advanced tools covered in this domain.

Core capabilities include:

1. ndarray - A fast, memory-efficient multidimensional array with sophisticated broadcasting capabilities
2. Vectorized operations - High-speed computations across entire datasets without explicit loops
3. File I/O - Utilities for reading/writing disk data and memory-mapped files
4. Integration - Seamless interfaces for code written in C, C++, and other languages

While NumPy doesn't provide extensive high-level data analysis functions directly, understanding array-based computing significantly enhances efficiency when working with tools like pandas.

Creating the ndarray Object

Overview

The ndarray (N-dimensional array) is NumPy's fundamental data structure - a fast, flexible container for large datasets. It enables mathematical operations across entire data blocks simultaneously.

Every ndarray possesses two key attributes:

• shape: A tuple specifying the size of each dimension
• dtype: An object describing the array's data type

All elements with in an array must share the same data type.

Standard convention: Import NumPy using import numpy as np. Avoid the from numpy import * approach as it pollutes the namespace.

Jupyter Notebook Shortcuts

• Esc - Enter command mode
• dd - Delete current cell
• a - Insert cell above
• b - Insert cell below

Creating Arrays

Using the array() Function

The simplest way to create an array is with the array() function, which accepts any sequence-like object (including other arrays) and returns a new NumPy array.

import numpy as np

One-Dimensional Arrays

a = np.array([1, 2, 3, 4, 5])
a

Checking properties:

a.dtype    # Returns: dtype('int64')
a.shape    # Returns: (5,)

Two-Dimensional Arrays

Wrap inner arays within an outer list:

b = np.array([[1, 2, 3, 4], [2, 4, 6, 8]])
b

Properties:

b.dtype    # Returns: dtype('int64')
b.shape    # Returns: (2, 4)

Character Arrays

c = np.array(list('abcdefg'))
c

Properties:

c.dtype    # Returns: dtype('<U1')
c.shape    # Returns: (7,)

Using zeros() and zeros_like()

The zeros() function creates arrays filled with 0.0 (float type by default). The zeros_like() function generates a zeros array matching the shape of a reference array without copying data.

d = np.zeros(5)
d.dtype    # Returns: dtype('float64')

e = np.zeros((2, 2))
e[0][0] = 100
e

f = np.zeros_like(e)
f

Note: The _like variants only copy the shape, not the actual values.

Using ones() and ones_like()

Creates arrays with all elements set to 1:

g = np.ones([3, 3])
np.ones_like(g)

Using empty() and empty_like()

Creates uninitialized arrays. Values are arbitrary and should not be relied upon:

h = np.empty([3, 3])
h

i = np.empty_like(g)
i

Identity Matrix with eye()

Creates an N×N identity matrix (1s on diagonal, 0s elsewhere):

np.eye(5)

Using arange()

The array equivalent of Python's range():

j = np.arange(10)      # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
j

k = np.arange(10, 20, 2)   # [10, 12, 14, 16, 18]
k

Data Types

The dtype attribute reveals the element type. Key types include:

Type	Code	Description
int8, uint8	i1, u1	8-bit signed/unsigned integers
int16, uint16	i2, u2	16-bit signed/unsigned integers
int32, uint32	i4, u4	32-bit signed/unsigned integers
float16	f2	Half-precision floating point
float32	f4 or f	Single-precision floating point
float64	f8 or d	Double-precision floating point
bool	?	Boolean type
object	O	Python object type
unicode_	U	Fixed-length Unicode string

Example: Creating arrays with different types

import numpy as np

int_arr = np.array([1, 2, 3, 4])
str_arr = np.array(list('abcdefg'))
bool_arr = np.array([True, False, False, True])

class Person:
    pass

obj_arr = np.array([Person(), Person(), Person()])

int_arr.dtype    # dtype('int64')
str_arr.dtype    # dtype('<U1')
bool_arr.dtype   # dtype('bool')
obj_arr.dtype    # dtype('O')

Type Conversion with astype()

orig = np.array([1, 2, 3, 4, 5])
converted = orig.astype(np.float32)
converted.dtype    # dtype('float32')

Float to integer truncates decimals:

float_arr = np.array([1.1, 2.2, 3.3, 4.4])
int_arr = float_arr.astype(np.int32)
int_arr    # [1, 2, 3, 4]

Numeric strings can be converted directly:

str_num = np.array(['10', '20', '30', '40'])
numeric = str_num.astype(np.int32)
numeric    # [10, 20, 30, 40]

Array Operations

Vectorization

Performing batch operations without loops is called vectorization. Arithmetic between arrays of different shapes is called broadcasting.

Reshaping Arrays

matrix = np.arange(9).reshape((3, 3))
matrix

Scalar Operations (Broadcasting)

a = np.arange(10)
a + 100    # [100, 101, 102, 103, 104, 105, 106, 107, 108, 109]

a * 10     # [0, 10, 20, 30, 40, 50, 60, 70, 80, 90]

matrix + 100

Boolean Operations

b = np.arange(10)
b > 2    # [False, False, False, True, True, True, True, True, True, True]

Random Number Generation

np.random.randint(0, 10, 5)              # Random 5 elements in [0, 10)
np.random.randint(0, 10, (3, 4))          # Random 3×4 matrix

Array-to-Array Operations

arr_a = np.random.randint(0, 10, 5)
arr_b = np.random.randint(0, 10, 5)
arr_a + arr_b    # Element-wise addition

mat_c = np.random.randint(0, 10, (3, 4))
mat_d = np.random.randint(0, 10, (3, 4))
mat_c + mat_d

row = np.random.randint(0, 10, 4)
mat_c + row      # Broadcasts across all rows

Note: One-dimensional arrays broadcast across rows when added to 2D arrays.

Array Indexing and Slicing

Basic Indexing

data = np.arange(25).reshape((5, 5))
data

Row access:

data[0]              # First row
data[0][1]           # First row, second element
data[:3]             # First three rows

Column slicing:

data[:3][:2]         # First three rows, then first two rows of result
data[:3, 2:]          # First three rows, columns from index 2 onwards

Critical distinction: NumPy slices return views, not copies. Modifications to a view affect the original array. This design optimizes memory and performance for large datasets. Use .copy() explicitly when you need an independent copy.

Fancy Indexing

grid = np.empty((8, 8))
for idx in range(8):
    grid[idx] = np.arange(idx, idx + 8)
grid

Selecting specific rows:

grid[[1, 3, 5]]              # Rows at indices 1, 3, 5

Nested selection:

grid[[1, 3, 5]][[1, 2]]      # From filtered rows, select rows 1 and 2

Column specification:

grid[[1, 3, 5], 0]           # First element from each selected row
grid[[1, 3, 5], :2]          # Elements 0-1 from each selected row

Element pairs:

grid[[1, 3, 5], [0, 1, 2]]   # Specific elements: (1,0), (3,1), (5,2)

Boolean Indexing

stats = np.random.randint(1000, 10000, (4, 3))
stats

Creating row index:

countries = np.array(['China', 'USA', 'Germany', 'France'])
countries == 'USA'

stats[countries == 'USA']    # Filter by country name

Creating column index:

categories = np.array(['Economy', 'Military', 'Population'])
col_idx = np.argwhere(categories == 'Military')[0][0]

stats[countries == 'France'][0][col_idx]    # France's Military value

Boolean Array with Slicing

labels = np.array(['aaa', 'bbb', 'ccc', 'ddd', 'eee', 'fff', 'ggg'])
dataset = np.arange(35).reshape((7, 5))

dataset[labels == 'ccc']                       # Single boolean array
dataset[labels == 'ccc', 2]                     # Boolean + integer
dataset[labels == 'ccc', 1:]                    # Boolean + slice

Negation and Combination

dataset[labels != 'ccc']
dataset[~(labels == 'ccc')]
dataset[~(labels > 'ccc')]

Combining conditions with & (and) and | (or):

dataset[(labels == 'aaa') | (labels == 'ccc')]
dataset[(labels > 'ddd') | (labels == 'aaa')]
dataset[(labels < 'eee') & (labels > 'bbb')]

Assigning Values with Boolean Masks

simple = np.arange(5)
table = np.arange(16).reshape((4, 4))
labels = np.array(['aaa', 'bbb', 'ccc', 'ddd'])

simple[simple > 2] = 666
simple

table[labels == 'aaa'] = 0
table[labels == 'bbb', 2:] = 1
table[(labels == 'ccc') | (labels == 'ddd')] = 2

The zip() Function

Iterates through multiple sequences pairwise:

for flag, row in zip([True, True, False, True, True, True, True], dataset):
    print(flag, row)

for flag, row in zip([True, True, False, True, True, True, True], dataset):
    if flag:
        print(flag, row)

np.where() Conditional Replacement

Syntax: np.where(condition, value_if_true, value_if_false)

sample = np.random.randint(-10, 10, (5, 5))
sample

Replace negative values:

np.where(sample < 0, 100, sample)

Nested conditions:

np.where(sample > -3, np.where(sample < 3, 100, sample), sample)