Python Exception Handling for a Reliable Import Pipeline

An import pipeline has at least two different failure categories: a particular record is invalid, or the import itself cannot run correctly. Treating both as the same problem leads either to fragile jobs that stop too early or silent failures that look successful. Set the boundary before writing try and except.

Catch an expected record-level failure

The example converts a collection of strings into non-negative integer quantities. A malformed quantity is an expected input problem, so the loop records it and continues. Errors outside that contract should not automatically be swallowed.

def parse_quantity(raw):
    value = int(raw)
    if value < 0:
        raise ValueError("negative quantity")
    return value

accepted, rejected = [], []
for record, raw in enumerate(["3", "bad", "-2", "0"], start=1):
    try:
        value = parse_quantity(raw)
    except ValueError as error:
        rejected.append({"record": record, "reason": str(error)})
    else:
        accepted.append(value)

assert accepted == [3, 0]
assert len(rejected) == 2

Keep the try block narrow

Only conversion and the related validation are inside try. If code that writes a report or updates a database is also placed there, the same handler may misclassify a storage failure as a bad row. Narrow blocks make the meaning of a caught exception easier to reason about.

Decide when the entire job must stop

A missing input file, unreadable destination or unavailable database may make the entire result unreliable. At the command-line boundary, report that failure and return an unsuccessful exit status. If the job writes output incrementally, define how an incomplete run will be identified so another process does not consume a partial result as complete.

Use cleanup without erasing the cause

Use with for files so handles close whether processing succeeds or fails. When translating a low-level exception into a domain error, use exception chaining to preserve its cause. Avoid except Exception: pass; it can hide a typo, an unexpected type or a broken dependency and leave you with plausible but incomplete data.

Give the operator a useful summary

Report accepted, rejected and total record counts together. State whether partial acceptance is permitted. A job can intentionally accept valid records while rejecting others, but it should never imply that every record was imported when that did not happen.

Practice and check your understanding

Add a rule that cancels the import if more than one record fails. Keep validation separate from the decision to publish output, and test that no final output is produced when the threshold is exceeded.

Continue learning

Explore the related Code Hacker course syllabus for structured learning. Then read Validate JSON Configuration in Python Before Using It to connect this exercise to the next skill.

Reference

Python errors and exceptions tutorial provides the underlying API or technical guidance. Check the documentation for the version you use; examples here are small learning exercises rather than complete production implementations.