How to evaluate a job data API: a practical trial checklist

Define the decision before opening a trial

A jobs API trial is easy to waste. You request the latest jobs, see familiar employers and attractive descriptions, and conclude that the data looks good. That proves the service can return a convincing sample. It leaves the important question unanswered: can it supply the jobs your customers need, at a cost your product can support?

Start with a one-page acceptance brief. Name the audience, geography, required employers, must-have fields and freshness expectations. A board for registered nurses in Ontario needs different evidence from a sales tool tracking technology companies across the United States. “Lots of jobs” is too broad to test. “Find current nursing requisitions from these 20 employers, with usable descriptions and application destinations” is a testable requirement.

Separate deal breakers from preferences. Missing a required employer might block launch. Missing salary on a posting whose employer never disclosed pay might be acceptable. Write those decisions down before comparing providers, so an impressive total does not quietly change your criteria.

Build a reference sample with a known denominator

Choose employers deliberately. Include your essential accounts, some smaller organizations, and several hiring systems. A sample composed entirely of large technology companies is unlikely to reveal the gaps a regional healthcare or retail product will encounter. Keep an unverified list separate from the employers you have actually checked.

Visit the original careers pages and record the requisition identifiers that meet your inclusion rules, together with source URLs and the observation time. If a page has several locations for one requisition, retain the locations but count the requisition once. Exclude talent communities or general applications when your product promises actual openings.

You need a complete reference inventory to calculate coverage for that employer. If you inspect only the first page of a 200-job board, label it a spot check. Do not turn those first 20 jobs into a claim about all 200. A useful small audit with clear limits is more credible than a large spreadsheet with an unknown denominator.

Calculate coverage, relevance and duplication separately

Suppose an employer has 40 eligible requisitions on its original careers site. The provider contains 32 of those unique requisitions. Sample coverage is 32 ÷ 40 = 80%. Keep the eight missing identifiers: they are more actionable than the percentage alone. Ask whether the gap reflects a missing employer board, a location filter, delayed discovery or unsupported details.

Now review 50 rows returned by your product search. If 45 actually fit your niche, relevance is 45 ÷ 50 = 90%. If those 45 relevant rows represent 42 distinct vacancies, three are redundant copies. Redundancy within the relevant set is 3 ÷ 45 = 6.7%. These measures answer different questions; combining them into one “quality score” too early hides the reason a provider succeeds or fails.

Report results by employer and role family as well as overall. A provider could cover one 500-job employer completely while missing nine small employers. The combined percentage would hide those omissions. For a product that promises named employers, that distinction matters.

Inspect the fields that determine whether a job is usable

Read full descriptions for a small, varied set of jobs. Check whether responsibilities, qualifications, working hours, benefits and application instructions survive. A response field called “description” could contain the full posting, a truncated summary or text extracted in the wrong order. Test the actual content in your own interface.

For salary, verify the amount, currency, period and meaning. A range of 40–50 per hour must not become 40–50 per year. A total compensation target should not be silently displayed as base salary. Record whether the value comes from the employer, a rule or an estimate. Accuracy and availability are separate: a missing value can be correct when the source supplies none.

Check application destinations too. A null apply_url may be acceptable when url points to a valid job page with an application action. A generic careers homepage usually gives applicants more work. Inspect extra source fields and provenance when your product needs details that are not standardized across employers.

Take a bounded technical sample

The downloadable Node.js collector searches for nursing jobs in Ontario, requests text descriptions and source fields, and preserves complete returned job objects in JSON. It stops after 100 returned records or 10 requests, counting duplicate rows against that budget. It also records whether it reached the end of the results, so a budget-limited sample cannot be mistaken for a complete inventory. Edit the country, province and title in the script for your own test.

Use Node.js 22 or later. Set THEJOBSAPI_KEY in your local shell environment, then run the command below. The script intentionally stops on an HTTP error instead of retrying blindly. Review the API error documentation and your remaining allowance before resuming.

node sample-jobs.mjs > sample.json

The pagination detail matters: /v1/jobs can return a short or empty page with a non-null next_cursor. Continue using that cursor until it is null or your sample budget is reached. Stopping on the first short page can make a provider look as if it has less coverage than it does.

The first 100 newest results are a convenience sample, not a random or representative one. Use them to test parsing and display. Use the deliberately selected employer inventory to judge coverage. The downloaded scorecard helps keep those two exercises separate.

• Search response and cursor semantics
• Error and quota handling

Measure freshness using the right clock

There are at least three useful times: when the employer says a job was posted, when the provider first discovers it, and when your application makes it available. A newly onboarded employer can produce many newly discovered jobs that were originally posted weeks earlier. Calling all of them “posted today” would mislead readers.

Track a small set of jobs over several observations. Record when you first see a source change and when it appears through the API. Without exact source publication times, you have an observation interval, not a precise latency measurement. If the employer changes at some point between your 09:00 and 10:00 checks, you cannot truthfully claim a 13-minute discovery delay.

Test removals as well as additions. Ask how the provider distinguishes a closed job from a failed source request, and how an updated job that leaves your geography is represented. Do not require a production employer to close a vacancy on demand; use documented fixtures to test your application, and retain real observations separately.

• Posting dates and discovery dates
• Change-feed event meanings

Estimate the monthly bill from operations

A monthly allowance usually measures API activity, not the number of rows sitting in your database. Write down initial import records, newly returned jobs, returned modifications, separate detail calls and reconciliation reads. Check request allowances independently. A large page reduces requests but does not necessarily reduce billed records.

Consider a hypothetical launch that imports 8,000 records, receives 2,000 new records, reads 3,000 modifications, fetches 1,000 separate details and spends 1,000 records on rechecks. That is 15,000 returned records before extra headroom. It is not 8,000 simply because the first import had 8,000 rows. A steady month may look different because the initial import is absent.

With The Jobs API, returned search records, job details and change events count toward the job allowance. Expired-ID responses do not contain billed job records, although their requests count. The ordered change feed is country-wide; it does not inherit your narrow search filters. A tiny niche board should compare filtered feeds with the cost of processing a whole-country stream before choosing its sync design.

• Current usage rules
• Choosing a sync strategy

Turn the trial into a decision memo

Finish with a short recommendation that another teammate could review: audience tested, observation dates, sample construction, coverage by essential employer, field defects, lifecycle results, expected usage and unresolved gaps. Include a few exact examples of failures. “Eight Ontario nursing requisitions missing from employer X at this timestamp” gives a provider something concrete to investigate.

Set the outcome to adopt, extend the trial with specific questions, or reject for the current use case. Do not average away a deal breaker with unrelated strengths. If you need a market or field the service does not provide, a larger allowance cannot fix that mismatch.

The Jobs API publishes this guide as one of the services you may evaluate. Its trial currently includes 500 jobs and 50 requests over seven days. Reserve some of that budget for details and lifecycle checks; the bounded script is only one part of the evaluation. Apply the same evidence standard to us as to every other provider.