Struggling with IP Geolocation API consistency: encountering stale data after database updates on our lookup tool

hey everyone, we've been running our 'IP Lookup Tool' for a while now, and it's been mostly smooth sailing. however, the core issue we're facing is pretty stubborn: data integrity for our IP geolocation results. it's becoming a real blocker for us.

we're seeing stale city/region data returned for specific IP addresses, even after we've pushed database updates or done direct API refreshes. it's like a propagation delay, but far too stubborn and inconsistent. for example, an IP will show as being in 'Old City' when the source commercial IP geolocation API clearly says 'New City' for that ip address lookup. this is happening even for well-known, frequently updated ranges.

our setup uses MaxMind GeoLite2 locally for quick lookups, backed by a custom wrapper around a commercial IP geolocation API for more granular data. we've checked all the usual suspects: TTLs on DNS, cleared Redis caches, purged Varnish, and even tried forced full database reloads on the MaxMind side. we even confirmed that our custom wrapper is indeed hitting the commercial API with no-cache headers when we force a refresh. it feels like there's some deeper, more persistent cache or synchronization issue that we're just missing.

here's a dummy console output showing the kind of discrepancy we're seeing after we've tried to refresh the data:

# IP Address Lookup: 192.0.2.1
--- Our Tool Output (After 'Refresh') ---
IP: 192.0.2.1
City: Old Town
Region: Old State
Country: US
Last Updated: 2023-01-01 10:00:00 UTC

--- Expected/Source API Output (Direct Query) ---
IP: 192.0.2.1
City: Newburg
Region: New State
Country: US
Last Updated: 2024-03-15 14:30:00 UTC

the Last Updated in our tool is clearly behind, even though we trigger refreshes. is anyone else dealing with such stubborn data freshness issues, especially with IP geolocation services or complex data pipeline synchronizations? it's driving us a bit mad. any insights or similar experiences would be super helpfull. anyone faced this before?