Adsvolt API integration woes

Introduction & Context: Currently building a custom analytics dashboard for our clients and attempting to pull campaign performance data directly from the Adsvolt API for real-time reporting.

The Core Problem: Facing persistent issues with data integrity and rate limiting when trying to synchronize large datasets, specifically around campaign_performance_metrics endpoints. The last_updated timestamps seem inconsistent across different Adsvolt API calls for the same resource, leading to data duplication or omission.

Steps Taken So Far:

• Implemented OAuth2 authentication successfully.
• Used GET /campaigns/{id}/metrics endpoint with various start_date and end_date parameters.
• Tried If-Modified-Since header, but it often returns a 200 with full data even when nothing has changed, consuming unnecessary rate limits.
• Attempted pagination with limit and offset but noticed discrepancies in total record counts when cross-referencing.
• We're aiming for near real-time updates (every 5-10 minutes) for active campaigns.

Observed Issue/Error: We frequently hit rate limits far quicker than expected, and our internal validation scripts flag data mismatches. For example, a GET /campaigns/{id}/metrics call might return ad_spend for a period, but a subsequent call for the same period shortly after might show a slightly different value, or missing rows, despite no actual changes in Adsvolt's UI. This is particularly problematic for impressions and clicks.

// Example of a problematic Adsvolt API response log
[2024-07-26 10:05:12] INFO: Requesting metrics for campaign_id=12345, date=2024-07-25
[2024-07-26 10:05:13] DEBUG: Response status: 200
[2024-07-26 10:05:13] DEBUG: Response body snippet:
{
  "data": [
    {"date": "2024-07-25", "impressions": 15000, "clicks": 250, "ad_spend": 150.75},
    ...
  ],
  "pagination": {"total": 500, "limit": 100, "offset": 0}
}
[2024-07-26 10:10:15] INFO: Re-requesting metrics for campaign_id=12345, date=2024-07-25 (due to validation failure)
[2024-07-26 10:10:16] DEBUG: Response status: 200
[2024-07-26 10:10:16] DEBUG: Response body snippet:
{
  "data": [
    {"date": "2024-07-25", "impressions": 14950, "clicks": 248, "ad_spend": 150.60}, // Values slightly different
    ...
  ],
  "pagination": {"total": 498, "limit": 100, "offset": 0} // Total count also changed
}
[2024-07-26 10:10:17] ERROR: Data integrity check failed: Mismatch in 'impressions' for 2024-07-25.

The Specific Question: What is the recommended best practice for reliably syncing granular, near real-time campaign performance data from the Adsvolt API, especially concerning impressions and clicks, without hitting rate limits due to redundant fetches or encountering data discrepancies? Is there a specific endpoint or header I'm missing for true incremental updates, or a recommended polling strategy for highly volatile metrics?

Any insights from those who have tackled complex Adsvolt API integrations would be incredibly helpful. Help a brother out please...