Rate Limits and Quotas
Rate Limits and Quotas
The Eternity library acts as a wrapper for the Google Custom Search JSON API. Consequently, your usage is governed by the quotas and rate limits defined by Google Cloud. Understanding these limits is essential for ensuring reliable data retrieval in your applications.
Google Custom Search API Limits
By default, the Google Custom Search API provides a mix of free and paid usage tiers:
| Tier | Limit | Cost | | :--- | :--- | :--- | | Free Tier | 100 search queries per day | Free | | Paid Tier | Up to 10k queries per day | $5 per 1,000 queries |
Once you exceed the 100-query daily limit on the free tier, the API will return an error, and the Eternity methods will fail to retrieve new data until the quota resets (typically at midnight Pacific Time) or billing is enabled.
Handling Quota Errors
When a rate limit is reached or a quota is exceeded, the search() and get_combined_text() methods will output the status code returned by Google.
Common status codes related to limits include:
429 Too Many Requests: You have sent too many requests in a given amount of time (Rate Limiting).403 Forbidden: You have exhausted your daily quota or the API key is invalid.
To handle these gracefully, you should wrap your calls in logic that checks for empty results:
from eternity import Eternity
client = Eternity(api_key="YOUR_API_KEY", engine_id="YOUR_ENGINE_ID")
results = client.get_combined_text("Python automation")
if not results:
# This may indicate a quota exhaustion or network error
print("No data retrieved. Check your Google Cloud Console quota.")
Managing High-Volume Requests
If your application requires more than 100 searches per day, consider the following strategies:
1. Enable Billing
To increase your limit beyond 100 queries/day, you must enable billing in the Google Cloud Console. This allows for up to 10,000 queries per day per project.
2. Implement Caching
Since Eternity performs real-time web requests, repetitive queries for the same terms will consume your quota quickly. We recommend implementing a local caching layer (such as sqlite3 or redis) to store results for a specific duration.
3. Request Throttling
If you are running Eternity in a loop or a multi-threaded environment, implement a delay between requests to avoid triggering temporary rate blocks.
import time
from eternity import Eternity
client = Eternity(api_key="API_KEY", engine_id="ENGINE_ID")
queries = ["Query 1", "Query 2", "Query 3"]
for q in queries:
data = client.search(q)
print(f"Retrieved {len(data)} results for {q}")
# Short delay to respect API stability
time.sleep(1)
Monitoring Usage
You can monitor your real-time usage and configure alerts for quota thresholds through the APIs & Services > Dashboard section of your Google Cloud Console. This is the most accurate way to track how many requests your Eternity implementation is consuming.