Problem

In many programming scenarios, you need to work with callback functions—such as event handlers or completion callbacks for asynchronous tasks. However, sometimes the callback needs access to additional state information during its execution, which isn't passed through the standard callback signature.

Basic Implementation

Consider a simple asynchronous execution functon that accepts a callback:

def execute_with_callback(operation, params, *, on_complete):
    output = operation(*params)
    on_complete(output)

def display_output(output):
    print('Received:', output)

def combine(a, b):
    return a + b

execute_with_callback(combine, (10, 20), on_complete=display_output)
execute_with_callback(combine, ('foo', 'bar'), on_complete=display_output)

The display_output() function only receives the result parameter. If you need the callback to access external variibles or maintain context-speccific data, this limitation becomes problematic.

Solution 1: Using Bound Methods

One approach is to replace a simple function with a bound method from a class that maintains internal state:

class CallbackWithCounter:
    def __init__(self):
        self.counter = 0

    def process(self, output):
        self.counter += 1
        print('[{}] Received: {}'.format(self.counter, output))

def execute_with_callback(operation, params, *, on_complete):
    output = operation(*params)
    on_complete(output)
    return 'completed'

def combine(a, b):
    return a + b

if __name__ == '__main__':
    processor = CallbackWithCounter()
    execute_with_callback(combine, (10, 20), on_complete=processor.process)
    execute_with_callback(combine, ('foo', 'bar'), on_complete=processor.process)

Solution 2: Using Closures

Alternatively, you can use a closure to capture and maintain state without defining a class:

def create_processor():
    counter = 0
    def process(output):
        nonlocal counter
        counter += 1
        print('[{}] Received: {}'.format(counter, output))
    return process

def execute_with_callback(operation, params, *, on_complete):
    output = operation(*params)
    on_complete(output)
    return 'completed'

def combine(a, b):
    return a + b

if __name__ == '__main__':
    handler = create_processor()
    execute_with_callback(combine, (10, 20), on_complete=handler)
    execute_with_callback(combine, ('foo', 'bar'), on_complete=handler)

Solution 3: Using Coroutines

A more advanced technique leverages coroutines to achieve the same result:

def create_coroutine_handler():
    counter = 0
    while True:
        output = yield
        counter += 1
        print('[{}] Received: {}'.format(counter, output))

def execute_with_callback(operation, params, *, on_complete):
    output = operation(*params)
    on_complete(output)
    return 'completed'

def combine(a, b):
    return a + b

if __name__ == '__main__':
    handler = create_coroutine_handler()
    next(handler)
    execute_with_callback(combine, (10, 20), on_complete=handler.send)
    execute_with_callback(combine, ('foo', 'bar'), on_complete=handler.send)