Jobs API vs building job scrapers: a practical cost comparison

What are you actually choosing to build?

A first scraper can be short. Fetch a page, find a title, save a description and put a row in a database. A reliable job-data product has a larger contract: discover the right employers, retrieve complete inventories, keep details current, recognize duplicates, distinguish outages from closed jobs and repair failures without losing source evidence.

Compare an API with that operating system, not with a successful one-off download. If your product promises live opportunities, collection needs an owner after launch. Each new hiring system adds mapping and verification work, and each employer can introduce exceptions that do not appear in the vendor’s simplest example.

The Jobs API’s proposition is that a small product team can buy maintained job access and focus on the applicant experience. It is not that custom engineering is always wasteful. If you serve only a handful of known employers or need a field no provider exposes, a direct integration can be the more appropriate choice.

Should you use an ATS feed before scraping HTML?

Yes, when an appropriate public or authorized feed exists. Greenhouse documents a public Job Board API with unauthenticated GET endpoints. Lever documents a public postings API. Those are useful starting points for supported employers because you can work with a published structure instead of selectors tied to page presentation.

An employer-specific feed still leaves discovery and maintenance work. You need to know which board belongs to which employer, whether you have every relevant posting, what each field means and how changes are represented. The existence of a clean endpoint does not give you a global employer directory or a complete multi-ATS inventory.

Evaluate each source within its access and use conditions. Do not make a business plan depend on solving challenges or logging into private systems to obtain public job data. If a required employer is unavailable, retain that as an explicit coverage gap and seek an appropriate integration or permission.

• Greenhouse Job Board API
• Lever postings API

Which responsibilities remain with each approach?

Buying reduces the collection surface your team owns. It does not remove the need to sanitize content, handle API errors, protect credentials, track quota or decide how jobs should appear to users. Include those remaining costs when evaluating an API; hiding them creates an unfair comparison.

Building gives you control over collection priorities and source-specific fields. That control is valuable when it supports a distinctive product requirement. It is less valuable when a team spends its limited time repairing parsers for the same basic fields a service already provides.

How do you calculate a fair monthly cost?

Estimate engineering time at a loaded hourly rate that fits your own team, then spread initial implementation over a defined planning horizon. Add recurring maintenance, infrastructure and paid data access. Keep opportunity cost visible, but do not mix a speculative revenue number into an otherwise concrete cash-cost comparison.

The following six-month example is illustrative, not a measured customer outcome. Both options include application integration effort. It assumes a $75 hourly engineering rate and the $50 entry API plan; a different volume or requirement could need a higher plan. Replace every workload assumption before using the table in a buying decision.

In this scenario, the difference is $860 per month. The assumptions create the result: it is not a universal saving. If you already operate reliable connectors, their initial cost is sunk and should not be charged again in a forward-looking decision. If one public feed takes almost no maintenance, the build estimate should reflect that.

Shared application hosting is excluded from both columns because the board needs it either way. Add source-specific bandwidth, monitoring, storage and licensing costs when they apply. For the API option, model backfills, detail calls and repeated reads, then replace the $50 line if that usage or the required feed feature calls for another tier.

The downloadable CSV keeps quantities, rates and the six-month allocation explicit. It contains ordinary values rather than spreadsheet formulas: edit the assumptions and recompute the monthly totals in your own workbook. This avoids presenting an illustrative budget as a ready-made forecast.

• Published API allowances
• Usage rules

What does a robust collector have to get right?

Completeness comes before expiration. If a source normally lists 200 jobs and one failed response contains ten, do not close the other 190. Your collector must distinguish a complete inventory from a partial page, an empty response, a refusal or an outage. Otherwise a transient failure becomes a false hiring signal.

Preserve the source payload that explains the normalized record. A salary parser can improve later; a discarded currency or nested location cannot be recovered from a title-only row. Keep source identity, fetch time and the relationship between listing and detail data so that a reviewer can trace a displayed field back to evidence.

Separate discovery from detail refresh. Repeatedly downloading an unchanged description wastes capacity, but a listing that omits compensation changes cannot prove that the detail page is unchanged forever. Define periodic verification and change detection according to what each source actually exposes.

Identity is another recurring cost. A requisition can have multiple postings, languages or locations. An employer can move between hiring systems. Store source identifiers and your matching evidence instead of merging on title alone. Keep duplicates, exclusions and expired records distinguishable so your reported growth means something.

When is an API the stronger business decision?

An API is compelling when the audience needs varied employers across supported US and Canadian sources, and source maintenance is not what makes your product distinctive. A regional job board can differentiate through local knowledge, useful filters and a good application experience while obtaining structured inventory through one documented integration.

The Jobs API supplies search, full text and HTML descriptions, available source fields and lifecycle endpoints. The entry plan is $50 monthly for 20,000 jobs and 10,000 requests. The separate expired-ID feed starts at that tier; the modified feed requires 50,000 jobs. An ordered changes endpoint is available on every plan, but its country-wide scope needs its own usage estimate.

Use the trial to verify essential employers and field availability before treating the API as a replacement. A supported ATS family does not establish complete coverage of every employer using it. Send exact missing requisitions and source URLs when you need a coverage review; a broad assurance is less useful than a reproducible check.

• Our source coverage
• Provider evaluation checklist
• Choosing update endpoints

When should you build or combine both?

Build when the scope is narrow and stable, the employer gives you a suitable feed, or a unique source field is essential. You may also need collection as a core capability for research or a business model that a provider’s agreement does not support. In those cases, own the operational obligations explicitly rather than treating the scraper as a finished script.

A hybrid can use an API for broad supported inventory and direct feeds for verified gaps. Assign one authoritative source per vacancy where possible, retain provenance and define how duplicate channels are reconciled. Count the distinct openings delivered to users, not the number of rows that two feeds happen to produce.

Set a review date for the build-versus-buy decision. Track maintenance hours, missing required employers, quality defects and actual API usage. The right choice can change as the product grows. A decision supported by those observations is more useful than an ideological preference for owning everything or outsourcing everything.