Common Python Data Analysis Mistakes: Side-by-Side Comparison of Key Methods

Sorting Operations

List Sorting

Python lists offer two distinct sorting approaches with different behaviors:

# sort() modifies the list in-place and returns None
numbers = [3, 1, 4, 1, 5]
result = numbers.sort()
print(result)  # None
print(numbers)  # [1, 1, 3, 4, 5]

# sorted() returns a new sorted list, leaving the original unchanged
numbers = [3, 1, 4, 1, 5]
new_list = sorted(numbers)
print(numbers)  # [3, 1, 4, 1, 5]
print(new_list)  # [1, 1, 3, 4, 5]

Nested Structures and Custom Sorting

For sorting nested lists, use the key parameter with sorted():

scores = [['Alice', 85], ['Bob', 92], ['Charlie', 78]]
sorted_scores = sorted(scores, key=lambda x: x[1], reverse=True)
# [['Bob', 92], ['Alice', 85], ['Charlie', 78]]

Note the difference between Python's built-in sort and NumPy's sort:

data = [[4, 2, 8], [1, 7, 3], [9, 5, 6]]
print(sorted(data))  # Only compares first element: [[1, 7, 3], [4, 2, 8], [9, 5, 6]]

import numpy as np
arr = np.array(data)
print(np.sort(arr))  # Sorts each row independently
# [[2 4 8]
#  [1 3 7]
#  [5 6 9]]

NumPy Array Sorting

a = np.array([[5, 2], [8, 4]])
print(np.sort(a))           # Sort along last axis (row-wise)
print(np.sort(a, axis=0))   # Sort column-wise

Pandas Sorting

# Series sorting - no 'by' parameter exists
series_data.sort_values(ascending=False, inplace=True)

# DataFrame sorting by column values
df.sort_values(by='column_name')
df.sort_values(by='column_name', ascending=False)
df.sort_values(by=['col_a', 'col_b'])

# Sort by index
df.sort_index(ascending=False)

Reverse Sorting

Environment Parameter
Lists reverse=True
NumPy/Pandas ascending=False

Iteration Patterns

Dictionary Iteration

data_dict = {'x': 10, 'y': 20, 'z': 30}
for item in data_dict.items():
    print(item)  # ('x', 10), ('y', 20), ('z', 30)

Series Iteration

for idx, val in series_obj.iteritems():
    print(f"Index: {idx}, Value: {val}")

DataFrame Iteration

# Row-wise iteration
for row_idx, row_data in df.iterrows():
    print(f"Row {row_idx}: {row_data}")

# Column-wise iteration
for col_name, col_data in df.iteritems():
    print(f"Column: {col_name}")

Element Removal Operations

Operation List Behavior Set Behavior
remove() Returns None; raises KeyError if missing Same as list
pop() Returns removed element; defaults to index 0; raises IndexError if empty Same as list
del del lst[2] syntax, no return value Not supported
discard() Not available Returns None; no error if missing
# List example
items = ['a', 'b', 'c']
items.remove('b')  # ['a', 'c'], returns None

# Set example
unique_items = {'a', 'b', 'c'}
unique_items.discard('d')  # {'a', 'b', 'c'}, no error
unique_items.remove('d')   # KeyError raised

Array Creation Functions

NumPy provides multiple array initialization methods:

np.ones((3, 4))      # 3x4 matrix of ones
np.zeros((2, 5))     # 2x5 matrix of zeros
np.eye(4)            # 4x4 identity matrix
np.full((3, 3), 7)   # 3x3 matrix filled with 7

Dropping Rows and Columns

Removing Rows

# By label name
df.drop(['row_1', 'row_2'])
df.drop(['row_1', 'row_3'], inplace=True)

# By position
df.drop(df.index[0])              # First row
df.drop(df.index[:3])             # First three rows
df.drop(df.index[[0, 2]])        # First and third rows

# Using filter conditions
selected_indices = df['column_a'].drop_duplicates().index
df_filtered = df.loc[selected_indices]

Removing Columns

# Using del - modifies in place
del df['column_a']

# Using drop - returns new DataFrame
df = df.drop(['column_b', 'column_c'], axis=1)
df.drop(['column_b', 'column_c'], axis=1, inplace=True)

Handling Missing Values

NumPy NaN Detection

a = np.array([np.nan, 3, 7, np.nan, 12, 8])
valid_entries = a[~np.isnan(a)]  # Use ~ for negation
# array([ 3.,  7., 12.,  8.])

Pandas NaN Detection

# Filter by null status
df[df['column_a'].isnull()]
df[df['column_a'].notnull()]

# Remove rows with missing values
df.dropna()                           # Drop rows where all values are NaN
df.dropna(subset=['column_name'])     # Drop rows where specified column is NaN

Common pd. Methods

  • pd.DataFrame / pd.Series - Object creation
  • pd.set_option - Display configuration
  • pd.read_csv / pd.read_excel - Data import
  • pd.concat - Combine DataFrames
  • pd.merge - Database-style joins
  • pd.qcut - Quantile-based discretization
  • pd.get_dummies - One-hot encoding
  • pd.pivot_table - Pivot tables
  • pd.crosstab - Cross-tabulation

Prefix vs Suffix Parameters

  • prefix: Adds prefix to column labels in pd.get_dummies()
  • suffixes: Specifies suffixes for overlapping columns in pd.merge()

Tags: python Pandas Numpy Data Analysis Common Mistakes

Posted on Tue, 11 Aug 2026 17:01:33 +0000 by mark_c