AWS eks documentation change
Summary
Added two new troubleshooting sections: 'Increased application sync time' and 'Applications repeatedly syncing or stuck out of sync' with diagnostic steps, common causes, and mitigation techniques
Security assessment
The changes focus on operational troubleshooting for Argo CD synchronization performance and reliability issues. No security vulnerabilities or weaknesses are addressed. While some configurations like ignoreDifferences and RespectIgnoreDifferences are mentioned, they're for managing configuration drift, not security features.
Diff
diff --git a/eks/latest/userguide/argocd-troubleshooting.md b/eks/latest/userguide/argocd-troubleshooting.md index 85624e7ff..769490278 100644 --- a//eks/latest/userguide/argocd-troubleshooting.md +++ b//eks/latest/userguide/argocd-troubleshooting.md @@ -7 +7 @@ -Capability is ACTIVE but applications are not syncingApplications stuck in "Progressing" stateRepository authentication failuresMulti-cluster deployment issuesNext steps +Capability is ACTIVE but applications are not syncingApplications stuck in "Progressing" stateRepository authentication failuresMulti-cluster deployment issuesIncreased application sync timeApplications repeatedly syncing or stuck out of syncNext steps @@ -197,0 +198,290 @@ For more on multi-cluster configuration, see [Register target clusters](./argocd +## Increased application sync time + +If your applications are syncing but taking longer than expected, use the following diagnostic steps to identify the cause. + +### Check last sync time + +Confirm the delay by reviewing when applications last synced: + + + # View last sync time for all applications + kubectl get application -n argocd -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.operationState.finishedAt}{"\n"}{end}' + + # View last sync time for a specific application + kubectl get application my-app -n argocd -o jsonpath='{.status.operationState.finishedAt}' + +### Check application conditions + +Review application conditions for reconciliation queue delays: + + + # Check conditions on an application + kubectl get application my-app -n argocd -o jsonpath='{.status.conditions}' + +### Check targetRevision configuration + +Applications using `targetRevision: HEAD` invalidate the manifest cache on every commit to the repository, which slows sync times: + + + # List applications using HEAD as targetRevision + kubectl get application -n argocd -o jsonpath='{range .items[?(@.spec.source.targetRevision=="HEAD")]}{.metadata.name}{"\n"}{end}' + +### Common causes + + * **No webhook configuration** : Without webhooks, Argo CD polls repositories at the default interval of 6 minutes. This delays detection of new commits. + + * **targetRevision set to HEAD** : Every commit to the repository invalidates the manifest cache. Argo CD then regenerates manifests on each reconciliation. + + * **Large or complex Git repositories** : Monorepos or complex Helm charts cause slow manifest generation because of the volume of files and templates to process. + + * **High number of Kubernetes resources in a single application** : Applications managing many resources cause slow cluster cache sync because Argo CD must track the state of each resource. + + + + +### Mitigations + + * **Configure Git webhooks** : Webhooks notify Argo CD immediately when changes are pushed, bypassing the default polling interval. For configuration steps, see [Argo CD considerations](./argocd-considerations.html). + + * **Use specific branch names or commit SHAs** : Set `targetRevision` to a branch name or commit SHA instead of `HEAD` to preserve the manifest cache between syncs. + + * **Split large monorepos** : Divide large repositories into smaller, focused repositories to reduce manifest generation time. + + * **Reduce resources per application** : Split applications with many Kubernetes resources into multiple smaller applications to reduce cluster cache sync time. + + * **Enable controller log delivery** : Controller logs provide visibility into reconciliation behavior and queue processing. For configuration steps, see [Access EKS Capabilities controller logs](./capabilities-controller-logs.html). + + + + +## Applications repeatedly syncing or stuck out of sync + +If your application syncs and then immediately goes `OutOfSync`, or if it stays stuck in a sync loop, the cause is usually drift between what Git defines and what exists in the cluster. Start with baseline diagnostics. + +### Gather diagnostic information + + + # View current sync and health status + argocd app get my-app + + # Show exact fields that differ between Git and live state + argocd app diff my-app + + # Check whether the app has ever reached a stable state + argocd app history my-app + + +The `argocd app diff` command is the most useful starting point. It shows you exactly which fields cause the application to appear out of sync. + +### Self-managed certificates cause drift + +Controllers such as cert-manager, OPA Gatekeeper, and KEDA generate certificates at runtime. These runtime values are not in Git, so Argo CD detects drift on every reconciliation. + +The symptoms are: + + * Application syncs, then immediately shows `OutOfSync` + + * The diff shows changes on a webhook `caBundle` field or a TLS Secret `data` field + + + + +To resolve this, add `ignoreDifferences` for the affected fields and enable `RespectIgnoreDifferences` in your sync options: + + + apiVersion: argoproj.io/v1alpha1 + kind: Application + metadata: + name: my-app + spec: + ignoreDifferences: + - group: admissionregistration.k8s.io + kind: ValidatingWebhookConfiguration + jsonPointers: + - /webhooks/0/clientConfig/caBundle + - group: "" + kind: Secret + jsonPointers: + - /data/tls.crt + - /data/tls.key + syncPolicy: + syncOptions: + - RespectIgnoreDifferences=true + +### Self-heal interrupts slow-starting workloads + +When `selfHeal` is enabled, Argo CD re-syncs the application when it detects drift. If your workload takes 30–60 seconds to start, the self-heal triggers before the workload becomes `Healthy`. With `prune` enabled, this might tear down partially-started resources. + +To resolve this, first fix the underlying drift (see the certificate scenario). If drift is not the cause, consider disabling self-heal for workloads that you manage exclusively through Git: + + + apiVersion: argoproj.io/v1alpha1 + kind: Application + metadata: + name: my-app + spec: + syncPolicy: + automated: + selfHeal: false + prune: false + +###### Note + +Self-heal backoff timing is an instance-level controller setting. If you need to adjust self-heal timing rather than disabling it, open an AWS Support case. + +### ApplicationSet or resource ownership collisions + +If two Applications or ApplicationSets manage the same Kubernetes resource, Argo CD shows a `SharedResourceWarning`. The resource never reaches a stable state. This commonly happens when a shared resource name is not scoped per environment or cluster. + +To resolve this: + + * Make the contended resource unique per owner. Add an environment or cluster suffix to the resource name. + + * When renaming an ApplicationSet, set `preserveResourcesOnDeletion: true` first to avoid destructive teardown of existing resources: + + + + + + apiVersion: argoproj.io/v1alpha1 + kind: ApplicationSet + metadata: + name: my-appset + spec: + syncPolicy: + preserveResourcesOnDeletion: true + +### Stuck deletion from resource finalizers + +If an application is stuck in `Terminating` state or shows "N objects remaining for deletion", the `resources-finalizer.argocd.argoproj.io` finalizer blocks removal until all managed resources delete. A managed resource with its own unprocessable finalizer blocks the deletion indefinitely. + +To confirm, list resources that have a deletion timestamp but have not been removed: + + + kubectl get all -n my-namespace -o json | \ + jq '.items[] | select(.metadata.deletionTimestamp != null) | {name: .metadata.name, kind: .kind, finalizers: .metadata.finalizers}' + +To resolve this: + + * Make sure the controller that owns the blocking finalizer is healthy and running. + + * If the owning controller is healthy but the finalizer is not being processed, remove the blocking finalizer from the stuck resource: + + + + + + kubectl patch resource-kind + resource-name -n my-namespace \ + --type json -p '[{"op": "remove", "path": "/metadata/finalizers/0"}]' + +### Failed sync does not auto-retry to the same revision + +After a sync to a specific revision fails, Argo CD does not auto-retry the same revision. This commonly happens because of a manifest defect such as a `ComparisonError` from a duplicate environment variable key. + +Confirm by checking the application status: + + + argocd app get my-app + # Look for: Operation: Sync Phase: Failed Revision: <sha> + +To resolve this, fix the manifest defect in your Git repository and push a new commit. Alternatively, trigger a manual sync: +