How to prevent expired jobs from staying in your app

Define what active means in your application

A source job can be active while your product should hide it. It might move outside your geography, change from remote to hybrid, or stop meeting your editorial criteria. Conversely, a job you previously removed can become active again. Model the source’s status separately from your application’s display decision.

Keep at least the source identity, country view, provider job ID, source status, local inclusion decision and last applied version or sequence. Preserve the reason for removal. “Closed by the source,” “excluded from this feed” and “does not fit this board” are different explanations that a saved-job page or support tool may need.

Do not use successful retrieval as a status check. The Jobs API’s detail endpoint can return expired records. An HTTP 200 says the request succeeded; the status field tells you whether that record is active.

Inventory every place a job can survive

Deleting a database row may not remove the job from the product. Your search index, a CDN page, an application cache, a saved recommendation and tomorrow’s email queue can all still contain it. List those copies before implementing the sync loop.

For each copy, choose an update mechanism and a failure policy. Search documents usually need replacement or deletion by stable key. A cached detail page may need invalidation. A saved-job page can retain a closed-state explanation. An unsent alert should be checked against current display status before sending.

Make that last check close to delivery. A job can close between building an email queue and sending it. Your queue should identify the job and resolve its current public state rather than treating a week-old rendered card as authoritative.

Choose filtered feeds or an ordered stream deliberately

The Jobs API provides two styles. Filtered feeds let you fetch new or modified records using their documented windows and filters, plus an expired-ID endpoint. The ordered /v1/changes stream provides created, updated, expired and excluded events with next_since and more. It is scoped to one country per request.

The ordered stream does not support your title, employer or province filters. Adding those search parameters does not make it a niche stream. A small board can consume many irrelevant country-wide events, and each returned change event counts toward its job allowance. Plan this cost before choosing the simpler-looking cursor loop.

The filtered-feed approach has its own tradeoffs: you must schedule and recover several windows, and confirm which endpoints your plan includes. Expired-ID responses count as requests but do not contain billed job records. Choose based on your audience’s size, expected change volume, recovery needs and budget.

• Recommended feed strategy
• Change-feed contract
• Allowance rules

Bootstrap without creating an unnoticed gap

A snapshot followed by “start listening now” can lose changes that happened while the snapshot was downloading. Decide where the incremental history begins before importing. If your provider supplies a snapshot-aligned checkpoint, retain it. The Jobs API’s documented change endpoint does not expose a “give me the latest cursor” operation; do not invent one or use a job-search cursor as a change cursor.

Starting /v1/changes with since=0 replays retained change history for that country. It is not a cheap way to request only future events. Estimate the cost and work before doing this on a new integration. For a small filtered product, the documented time-window strategy may be more appropriate than replaying a whole country.

For a filtered backfill, record its start time, retain all page continuations, and recover overlapping new, modified and removal windows through completion using the documented endpoints. Upserts and removals must be safe to repeat. The 6m active feed is bounded to records first collected within 183 days; determine whether that boundary fits your required inventory. A bounded window should not be described as an all-time snapshot.

Do an explicit reconciliation before declaring the import complete. Compare a manageable, verified scope with the provider’s current active set, and investigate mismatches. Do not delete a whole local collection because one timed-out or incomplete source query returned fewer rows.

Commit job changes and the checkpoint together

The checkpoint is a statement that everything through that position has been applied. If you save it first and the process crashes before writing jobs, the next run skips unfinished work. If you write jobs first and crash before saving the cursor, the next run repeats work. The second case is recoverable when writes are idempotent.

Use a database transaction to commit local job decisions, pending delivery work and the new checkpoint as one operation. A transaction provides an all-or-nothing boundary for those local writes. It does not include an external search service or CDN; queue durable work for those systems and retry it separately.

The following is an implementation outline, not a ready-to-run SDK. Fetching occurs outside the transaction. The compare step prevents two workers from applying batches based on the same old cursor. Use one active consumer per stream or implement an equivalent lease.

expected = loadCheckpoint(stream)
batch = fetchChanges(country, expected)
validateResponse(batch)
decisions = resolveCurrentStateAndEligibility(batch)

BEGIN TRANSACTION
  lock checkpoint row for stream
  if stored checkpoint != expected: ROLLBACK and reload
  apply decisions by stable job key
  enqueue versioned search/cache work
  save checkpoint = batch.next_since
COMMIT

if batch.more: request the next batch
otherwise: schedule the next poll

• PostgreSQL transaction semantics

Understand replay and current-state records

An ordered event is not necessarily a frozen historical job document. In The Jobs API, the change entry identifies an event while the included job comes from the current stored record. Do not use a replayed payload as proof of exactly what a job said at the event timestamp.

This matters during a long catch-up. A job may have expired and later reopened before you read the old expiration event. For robust reconciliation, treat events as signals that a job needs attention. Coalesce repeated IDs within a batch and resolve current state when event kind and job status disagree or the included detail is insufficient. A detail fetch can return the current full record, at the cost of another returned job record.

Re-evaluate your niche filters on updates. An active job that now requires office attendance should leave a remote-only board. Keep sequence values as strings in checkpoints and avoid assuming they are consecutive: a country-scoped stream can have gaps in its sequence values. Continue using next_since, not a number you calculate yourself.

Make downstream retries safe

An outbox is a table of pending actions committed with your job changes. A separate worker updates search and caches, then marks each action done. If the search service is unavailable, the database transaction can still finish and the pending action survives a restart.

Ordering matters. Imagine an older “publish” action retries after a newer “remove” action succeeded. Without a version check, the older action can resurrect a closed job. Store a projection version per job, and ensure consumers either reject stale versions or read and publish the latest authoritative projection when processing the task.

Include the country and provider in your stream and job keys. Maintain independent Canadian and US checkpoints. Reusing a cursor across countries can skip data, even when both feeds use numeric-looking sequence strings.

Test failures with expected outcomes

Test the workflow in a disposable environment with controlled fixtures. You should not need a real employer to close a job on demand. Begin with an active job, update it out of scope, expire it, then reactivate it. Check the public projection and each downstream copy after every step.

Inject crashes around the transaction boundary and retries around the delivery boundary. The important invariant is that the committed checkpoint never gets ahead of durable local work, and older downstream work never overwrites a newer public state.

Monitor useful freshness rather than worker uptime

A running process can still be hours behind. Track last successful poll time, checkpoint progress while more is true, oldest pending outbox action, repeated errors and a small sample of public records checked against current source state. A flat checkpoint during an empty period is normal; a flat checkpoint while the feed reports more work is not.

Define an application freshness target that includes collection, your polling interval, backlog processing and cache delivery. Polling every minute does not make the upstream source one minute fresh. Measure the whole path and make your user-facing timestamps reflect what you actually know.

Keep an incident playbook: pause on quota exhaustion, retain the checkpoint, fix the cause, resume, drain delivery work and reconcile a sample. If your chosen feed’s retained history cannot cover the outage, plan a controlled re-import rather than silently calling the database current.