Bulk IP Lookup API: Rate Limit Architecture
A bulk IP lookup API lets you resolve many addresses in one request instead of one call per address, and the architecture around it comes down to three constraints: the maximum batch size, the request budget you are allowed to spend, and how well you avoid asking the same question twice. This post walks through a practical design, using ipto's bulk endpoint for the concrete numbers.
What the bulk IP lookup endpoint accepts
ipto's bulk lookup is a POST to /v1/ip/bulk with a JSON array of IP strings. The request takes up to 100 addresses, duplicates are removed before processing, and the response is an object keyed by IP address, each entry shaped like a single lookup result with country, city, ASN, organization and the is_vpn, is_tor, is_abuser, is_datacenter and bogon fields. Bulk is a paid-tier feature, available on Pro and Enterprise, while the Free tier covers single lookups at 1,000 requests per day with no signup. Check the API documentation for the current request format.
The important accounting detail is that bulk is charged per item. A batch of 100 addresses counts as 100 against your daily budget, exactly as 100 individual lookups would. Batching therefore does not stretch your quota. What it saves is round trips, connection setup and client-side orchestration, which is often the real bottleneck when you are enriching a log file or a customer list.
Design the pipeline around the budget
Because every address spends budget, the cheapest request is the one you never send. Start by filtering. Private and reserved addresses will never have meaningful geolocation, and you can drop them locally or classify them with the bogon IP checker before they cost anything. Deduplicate the remaining set, because production logs are dominated by a small number of repeat visitors, crawlers and NAT gateways. A hash set in front of the batcher is usually the highest return change you can make.
Then add a cache keyed by address with a time to live that matches how fast the data you care about changes. Country and ASN mappings are stable for long periods, so days is reasonable. Reputation flags change faster, since ipto's threat feeds refresh every 6 hours and its IP and ASN datasets refresh daily, so caching a verdict for longer than a few hours adds staleness without saving much. Keep separate TTLs per field if you can, or cache the whole result at the shorter TTL.
- Filter bogon and obviously invalid input locally.
- Deduplicate within the job and against the cache.
- Chunk the remaining addresses into groups of at most 100.
- Send chunks with bounded concurrency rather than all at once.
- Write results to the cache and the destination store, keyed by address.
Concurrency, backoff and the daily window
A daily budget is a fixed window, so the failure mode to design for is exhausting it partway through a job. Estimate cost before you start: the number of unique, non-bogon addresses is the number of requests the job will spend. If that exceeds what remains today, split the job across windows, prioritize the most valuable addresses first, or fall back to single lookups only for the highest-risk ones. Persist progress so a restarted job resumes from the last completed chunk rather than from the beginning.
Use modest concurrency. A handful of parallel chunks is almost always enough, since each chunk already carries 100 lookups, and unbounded fan-out only makes failures harder to reason about. On a transient error or an over-budget response, retry with exponential backoff and jitter, and treat an exhausted budget as a signal to stop and resume later, not something to retry in a tight loop. Make every write idempotent so a retried chunk cannot double-insert rows.
Choosing between bulk and single lookups
For real-time paths such as a sign-up form or a checkout, use a single lookup per event. The latency is low, the cache absorbs repeats, and you only spend budget on addresses that actually arrive. Bulk is for offline work: backfilling a table, scoring a day of logs, or auditing an allowlist. If your goal is risk scoring rather than raw enrichment, the fraud score checker combines IP, email, phone and User-Agent signals into a single 0 to 100 score, which can replace several separate calls. For a longer comparison of free-tier limits across providers, see our free-tier rate comparison.
Finally, log your own usage. Track requests spent per job, cache hit rate and the share of addresses dropped as bogon or duplicate. Those three numbers tell you whether the architecture is working long before a budget alarm does.
Before you script a large batch, try a handful of addresses by hand: the ASN lookup shows which network owns a range you keep seeing in your logs, which often lets you handle a whole network with one decision instead of thousands of single lookups.