Half the client bank statements will not open in the pipeline at all

A queue of client files failing at the first step is not one problem. It is four, needing four different responses: a file encrypted with a user password, a file that opens normally but forbids text extraction, a file damaged in transfer, and a photograph of a statement with no text in it at all. Retrying the job helps none of them. Classifying the failure resolves most without contacting anybody.

Password protected bank statement PDF automation is where firms lose the most time, because the obvious response — ask the client — is the slowest possible one and is often unnecessary. A large share of files that look locked are not locked to a reader at all; they are locked to a copier, which is a different flag with a different answer.

Classify the failure before anyone retries or re-requests

1. Capture the exit code and the error string, not just failed. The Xpdf and poppler command-line tools separate these cleanly: exit code 1 means the file could not be opened, exit code 3 means the problem was permissions. Two distinct routes, one boolean away from being indistinguishable in your logs.
2. Ask the file what it thinks it is. A metadata probe such as pdfinfo reports whether the document is encrypted and, when it is, prints the print and copy permissions it declares. Store that output on the job record; it is the evidence for whichever branch you take next.
3. Try opening with an empty user password. Files carrying only an owner password open with no password at all, and a pipeline that treats encrypted as locked will send a needless request to a client for a file it can already read.
4. Count extractable characters per page. A file that opens, reports 8 pages and yields 0 characters is an image, not a locked document, and belongs on an OCR path rather than in a password queue.
5. Compare the stored byte length against what the sender declared, where the channel gives you one. A file truncated by an upload that dropped mid-transfer produces a parser error that looks exactly like corruption, and re-requesting it succeeds first time.

The password the bank derives from something the client knows

Statements emailed by a bank are frequently protected with a password derived from data the account holder already has: a date of birth, part of an identity or account number, a customer reference. The rule is usually stated in the covering email and varies by bank, by product and by country, so a firm collecting statements for 40 clients across 15 banks meets 10 or more conventions in a single month.

Two responses are defensible and one is not. Defensible: ask the client to supply the file again from the bank's own portal, where downloads are typically unprotected, or ask for a structured export in place of the PDF, which removes the whole class of problem and improves everything downstream. Not defensible: building anything that derives or guesses passwords from client data you hold. Even where it would work, it converts a document-processing system into a credential-guessing system, and that is not a conversation any firm wants to have with a client or an insurer.

The file opens perfectly and the text is fenced off

A PDF can carry restrictions without carrying a password that a reader needs. The document declares what it permits — printing, copying, modification — and a conforming reader honours those declarations. The distinction matters operationally, because these files display fine on a partner's screen while every extraction job against them fails, which is exactly the pattern that gets logged as a mysterious parser bug.

The tooling makes the boundary visible. The poppler utilities document an owner-password option whose stated effect is to bypass all security restrictions, and they return a permissions-specific exit code when they refuse. So the technical question is trivial and the real question is not technical at all: does the firm have authority to override a restriction the bank set on a document the client owns? Get that answer in writing from the client, record it against the engagement, and take local advice on whether anything further applies in your jurisdiction. This page is engineering guidance, not legal advice.

Damaged in transfer, and the ones that were never text

• Truncated uploads. A mobile client on a weak connection produces a file that is structurally a PDF for most of its length and then stops. The parser error names a broken cross-reference or an unexpected end of file, and the fix is to ask for the same file again rather than to change the parser.
• Rewritten by something in the middle. Mail gateways, archiving tools and chat apps re-encode attachments, and a re-encode can flatten text to images or strip an embedded font. The tell is that the client's copy extracts fine and yours does not.
• Printed and rescanned. A clean digital PDF, printed, signed and scanned back, is now an image at whatever the device defaulted to — often 150 or 200 dpi, against the 300 dpi recognition engines generally want. Nothing is broken; everything is worse.
• A photograph in a PDF wrapper. Phone scanning apps produce valid documents with 0 characters of text. They open, they page correctly, and they need OCR — plus the quality gate that belongs at capture rather than in the queue.
• Image-only files degrade more than the amounts. Recognised narration strings are noisier than digital text, which means counterparty names come through inconsistently and land you in one supplier arriving under six different names even when every figure is read correctly.

Where a client's key lives, and for how long

The moment a firm accepts a password it has accepted a custody problem, so the design should shorten how long that lasts. Take the credential on a channel separate from the document, hold it for the 1 job that needs it, and delete it on completion. Where statements arrive monthly and the password is stable, resist storing it in the client record: a secrets store with scoped access and an expiry measured in hours is the least you should accept, and asking again each period is better.

• Never in the ticket. A password pasted into a support ticket, an email thread or a shared inbox is now in every backup, every export and every search index the firm owns.
• Never in the same message as the file. If both arrive together, the credential adds no protection and should be treated as already exposed.
• Log the use, not the value. The job record should say a credential was supplied, by whom, at what time and for which document — and never contain the credential.
• Decrypt once, into a working copy. Re-supplying the password to every downstream step multiplies the number of places it can be captured.
• Say what you actually do. We hold no security certifications and do not claim any; what a firm can reasonably ask of a build is that the handling design is written down, testable and reviewed by its own advisers.

A pipeline that can read a restricted document is an engineering fact. Whether it should is a question about authority, and the answer belongs in the file, not in the code.

What the failed-file queue has to hold to be useful

The queue is the product here. A file that could not be opened still needs a stable internal reference from the moment it arrived, because everything afterwards — the request to the client, the retry, the eventual match to an engagement — hangs off that identity, the same argument made for the lifecycle of an invoice reference number. Record the failure class, the probe output, the client and engagement, the period the file appears to cover, every attempt with its timestamp, and what was asked of whom.

Then measure it by class rather than in aggregate. A firm with 300 failures a month made up of 240 image-only files has an intake problem to solve at capture; the same 300 made up of 240 permission-restricted files has a client-authority conversation to have once and never again. One number cannot tell those apart, and building the view that can is ordinary internal tools and ops work.

Opening the file is the beginning of the checks, not the end

A decrypted statement is still an unvalidated statement, and the gates behind this step catch the real damage. Statements must prove their own completeness before anything is posted; invoices must satisfy their own arithmetic, which is where line items that do not sum to the invoice total get caught. Neither check runs on a document the pipeline never opened, which is why a large queue here makes everything downstream look healthier than it is.

One boundary case is worth naming. A client who sends 1 file containing 6 months of statements has not sent a locked document at all, and it will fail differently: the file opens, and the splitter has to decide where 1 statement ends and the next begins, which is a batch scan splitting into the wrong number of documents. Keep the two queues apart, or one class of failure will mask the other in every report you produce about extraction and classification of source documents inside the accounting and tax systems we build.

• document intake
• bank statements
• PDF handling
• client data