Google API Keys Remain Active After Deletion

The findings reveal that Google API keys do not become inactive immediately upon deletion, allowing attackers an opportunity to exploit them for a limited window of time.

Joe Leon, a researcher at Aikido Security, a Belgian startup, conducted an analysis of the revocation window — the duration between the deletion of a key and its final successful authentication — for Google's API keys. In a recent blog post, Leon emphasized that Google Cloud Platform (GCP) users typically expect that the removal of an API key will halt its access instantaneously; however, this is not the reality.

His tests indicated that the median revocation window lasted approximately 16 minutes, with some instances extending up to 23 minutes. Leon described this duration as "an incredibly long time" for API keys to retain authentication capabilities.

The implications of these findings are significant for organizations. Leon explained, "An attacker with access to your deleted key can continue to send requests until one of them successfully reaches a server that has not yet processed the revocation. If Gemini is activated within the project, they could access uploaded files and exfiltrate cached conversations." Furthermore, he noted that while the GCP console would no longer display the key, it would not indicate that the key remains operational. This situation places complete reliance on Google’s infrastructure to eventually process the deletion.

Google API Key Revocation Windows Vary

Leon shared with Dark Reading that his interest in exploring GCP's revocation windows was sparked by earlier research conducted by Eduard Agavriloae, the co-founder of Offensai, which examined revocation delays in AWS credentials. Notably, Agavriloae's research indicated a mere four-second window for AWS, prompting an appropriate response from AWS on the matter.

"Four seconds was enough to matter on AWS," he remarked.

In stark contrast, the revocation windows for Google's API keys were substantially longer. The Aikido research team completed 10 tests over a two-day period, generating virtual machines (VMs) in various GCP regions, deleting the API keys, and sending up to five authenticated requests per second to assess the duration of the keys' functionality following deletion.

The results revealed highly variable outcomes. In one trial, the authentication success rate after one minute recorded as high as 79%, while another recorded a mere 5%. Additionally, the research team noted significant variations in success rates depending on the geographical region of the VMs.

For instance, test results illustrated that VMs located in GCP's asia-southeast1 region exhibited a median authentication success rate of just 22% after one minute. In contrast, the success rates for the us-east1 and europe-west1 regions were around 49%. Leon noted an unexpected observation: VMs situated further from the US processed deletion requests more rapidly, presenting a challenge in understanding the underlying factors.

"Google's request routing is more sophisticated than a direct relationship between VM region and server location," Leon analyzed. "A VM in Singapore may not directly communicate with servers based there." However, he emphasized that the pattern remained consistent across trials, suggesting an underlying influence of regional infrastructure, caching, or routing preferences.

Regardless of the cause, Leon contended that the regional discrepancies are influenced by the origin of the request rather than the client's geographic position.

API Key Deletion Delays Complicate Incident Response

According to Aikido's report, GCP's user interface (UI) for deleting keys asserts that, "Once deleted, it can no longer be used to make API requests." Leon criticized this assertion as misleading, highlighting the lack of clarity regarding the complete revocation timeline for an API key.

Leon communicated to Dark Reading that the prolonged revocation windows and unpredictable authentication success rates present challenges for incident response (IR) personnel dealing with potential breaches.

"This undermines the common assumption IR teams hold regarding leaked credentials," he noted. "It is generally expected that initiating a 'Delete' or 'Revoke' action immediately deactivates the credential. Consequently, IR teams now must account for the reality that a deleted credential could still be exploited by attackers."

To mitigate risks, Aikido recommended that security teams and IR officers observe a precautionary 30-minute window following the deletion of Google API keys. Organizations are also encouraged to monitor their API requests categorized by credential through the "Enabled APIs and services" section of the GCP console and to scrutinize API usage linked to credentials. Leon cautioned, "If you notice unexpected activity from a credential post-deletion, it may indicate active exploitation."

Aikido reported these findings to Google; however, the response was to close the report with a "won’t fix" designation, as indicated in the blog post. Dark Reading reached out to Google for comment on Leon's findings, but no response was received by press time.

Leon pointed out that Google successfully implements quicker revocations for alternative credential types, such as service account deletions, which propagate throughout the platform in about five seconds, and the newer API key format introduced by Gemini that can be fully revoked within approximately one minute. This indicates that reducing the revocation window for Google API keys is technically feasible.

"Managing distributed systems at the scale of Google's infrastructure is complex, and this is not a critique directed at the GCP IAM team," Leon concluded. "However, a 23-minute revocation window is fundamentally inconsistent with user expectations of a delete function."