Python categorizes sequence types in to several families. Container sequences (like list, tuple, and deque) can hold items of mixed types. Flat sequences (like str, bytes, bytearray, and array.array) store homogeneous data and are iterable with for loops. Mutable sequences (list, deque, bytearray, array) allow in-place modification, while immutable sequences (str, tuple, bytes) do not.
Abstract Base Classes for Sequences
Python's collections.abc module defines the inheritance hierarchy for sequences. The public API includes Sequence, MutableSequence, and ByteString among others.
__all__ = ["Awaitable", "Coroutine",
"AsyncIterable", "AsyncIterator", "AsyncGenerator",
"Hashable", "Iterable", "Iterator", "Generator", "Reversible",
"Sized", "Container", "Callable", "Collection",
"Set", "MutableSet",
"Mapping", "MutableMapping",
"MappingView", "KeysView", "ItemsView", "ValuesView",
"Sequence", "MutableSequence",
"ByteString",
]
Sequence extends Reversible and Collection. Reversible requires implementing __reversed__. Collection combines Sized (for __len__), Iterable (for __iter__), and Container (for __contains__). To create a custom immutable sequence, you must override __getitem__ and __len__ at minimum.
class Sequence(Reversible, Collection):
__slots__ = ()
class Reversible(Iterable):
__slots__ = ()
@abstractmethod
def __reversed__(self):
while False:
yield None
class Collection(Sized, Iterable, Container):
pass
class Sized(metaclass=ABCMeta):
__slots__ = ()
@abstractmethod
def __len__(self):
return 0
class Iterable(metaclass=ABCMeta):
__slots__ = ()
@abstractmethod
def __iter__(self):
while False:
yield None
class Container(metaclass=ABCMeta):
__slots__ = ()
@abstractmethod
def __contains__(self, x):
return False
MutableSequence inherits from Sequence and adds abstract methods for mutation: __setitem__, __delitem__, insert, append, clear, reverse, extend, pop, remove, and __iadd__.
class MutableSequence(Sequence):
__slots__ = ()
@abstractmethod
def __setitem__(self, index, value):
raise IndexError
@abstractmethod
def __delitem__(self, index):
raise IndexError
@abstractmethod
def insert(self, index, value):
raise IndexError
def append(self, value):
self.insert(len(self), value)
def clear(self):
try:
while True:
self.pop()
except IndexError:
pass
def reverse(self):
n = len(self)
for i in range(n//2):
self[i], self[n-i-1] = self[n-i-1], self[i]
def extend(self, values):
for v in values:
self.append(v)
def pop(self, index=-1):
v = self[index]
del self[index]
return v
def remove(self, value):
del self[self.index(value)]
def __iadd__(self, values):
self.extend(values)
return self
+ vs += vs extend
The + operator requires both operands to be of the same type.
a = [1, 2]
b = a + [3, 4] # Works: b = [1, 2, 3, 4]
# b = a + (3, 4) # TypeError: can only concatenate list (not "tuple") to list
+= works with any iterable:
c = [3, 4]
c += (1, 2) # c -> [3, 4, 1, 2]
c += 'hello' # c -> [3, 4, 'h', 'e', 'l', 'l', 'o']
Internally, += calls __iadd__, which extends the list by iterating:
def __iadd__(self, values):
self.extend(values)
return self
extend modifies the list in place and returns None. It accepts any iterable:
d = [5, 6]
a.extend(d) # Works
a.extend((5, 6)) # Also works
Implementing a Sliceable Custom Sequence
You can create a class that supports slicing by implementing __getitem__, __len__, and optionally __iter__ and __contains__. The example below builds an immutable-like Group that holds a list of staff members.
import numbers
class Group:
def __init__(self, group_name, company_name, staffs):
self.group_name = group_name
self.company_name = company_name
self.staffs = staffs
def __reversed__(self):
self.staffs.reverse()
def __getitem__(self, item):
cls = type(self)
if isinstance(item, slice):
return cls(group_name=self.group_name,
company_name=self.company_name,
staffs=self.staffs[item])
elif isinstance(item, numbers.Integral):
return cls(group_name=self.group_name,
company_name=self.company_name,
staffs=[self.staffs[item]])
def __len__(self):
return len(self.staffs)
def __iter__(self):
return iter(self.staffs)
def __contains__(self, item):
return item in self.staffs
staffs = ["bobby1", "imooc", "bobby2", "bobby3"]
group = Group(company_name="imooc", group_name="user", staffs=staffs)
print(group[0]) # Uses __getitem__ with int
print(group[:2]) # Uses __getitem__ with slice
reversed(group)
for user in group:
print(user)
This class correctly handles integer indexing, slicing (returning a new Group), iteration, containment checks, and reversal.
Maintaining Sorted Sequences with bisect
The bisect module helps keep a sorted list sorted while inserting new elements.
import bisect
items = []
bisect.insort(items, 3)
bisect.insort(items, 2)
bisect.insort(items, 5)
bisect.insort(items, 1)
bisect.insort(items, 6)
print(items) # [1, 2, 3, 5, 6]
You can also use a deque from collections, but bisect works with any mutable sequence that supports __getitem__ and insert.
When Not to Use a List
list can hold heterogeneous data, but for homogeneous numeric data, array.array is more memory-efficient and faster.
import array
# Create an integer array
my_array = array.array("i") # "i" stands for signed int
my_array.append(1)
print(my_array) # array('i', [1])
# my_array.append("abc") # TypeError: an integer is required
Use array when you need to store a large collection of numbers of the same type. Use deque for fast appends and pops from both ends.
Comprehensions and Generator Expressions
List comprehension
squares = [x * x for x in range(6)]
print(squares) # [0, 1, 4, 9, 16, 25]
# Cartesian product
pairs = [(a, b) for a in [1, 2] for b in [3, 4]]
print(pairs) # [(1, 3), (1, 4), (2, 3), (2, 4)]
def to_str(x):
return str(x)
strings = [to_str(x) for x in range(6)]
print(strings) # ['0', '1', '2', '3', '4', '5']
Dictionary comprehension
def process(k):
return str(k)
d = {process(k): k for k in range(5)}
print(d) # {'0': 0, '1': 1, '2': 2, '3': 3, '4': 4}
Generator expression
my_dict = {"key1": "bobby1", "key2": "bobby2"}
gen = ((k, v) for k, v in my_dict.items())
print(gen) # <generator object <genexpr> at 0x...>
for pair in gen:
print(pair)