kubectl describe pod --selector app.kubernetes.io/component=resultCache -n gooddata-cn Name: gooddata-cn-result-cache-7f6bdbcf6f-9hphl Namespace: gooddata-cn Priority: 0 Node: ip-10-163-173-89.us-west-2.compute.internal/10.163.173.89 Start Time: Wed, 01 Nov 2023 15:45:25 +0000 Labels: app.kubernetes.io/component=resultCache app.kubernetes.io/instance=gooddata-cn app.kubernetes.io/name=gooddata-cn pod-template-hash=7f6bdbcf6f Annotations: prometheus.io/path: /actuator/prometheus prometheus.io/port: 9041 prometheus.io/scrape: true Status: Running IP: 10.163.190.106 IPs: IP: 10.163.190.106 Controlled By: ReplicaSet/gooddata-cn-result-cache-7f6bdbcf6f Containers: result-cache: Container ID: containerd://9a7b4e95e426b6004b6309c7dbc99b061a30a5687dd85c75f9dc7e175fb443f7 Image: artifactory.cwantools.io:5000/gooddata/result-cache:2.5.1 Image ID: artifactory.cwantools.io:5000/gooddata/result-cache@sha256:e9cea80137c286f91576d78af899294c192474e2a833349fffe568ccb50ef46a Ports: 6567/TCP, 9041/TCP Host Ports: 0/TCP, 0/TCP State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 1 Started: Wed, 01 Nov 2023 16:34:40 +0000 Finished: Wed, 01 Nov 2023 16:35:14 +0000 Ready: False Restart Count: 13 Limits: cpu: 750m memory: 2Gi Requests: cpu: 750m memory: 2Gi Liveness: http-get http://:9041/actuator/health/liveness delay=30s timeout=5s period=10s #success=1 #failure=5 Readiness: http-get http://:9041/actuator/health/readiness delay=30s timeout=10s period=10s #success=1 #failure=5 Startup: http-get http://:9041/actuator/health/liveness delay=30s timeout=5s period=10s #success=1 #failure=12 Environment: JDK_JAVA_OPTIONS: -XX:+ExitOnOutOfMemoryError -XX:+UseG1GC -Xss384k -XX:+UseStringDeduplication -XX:MinHeapFreeRatio=15 -XX:MaxHeapFreeRatio=25 -XX:AdaptiveSizePolicyWeight=50 -XX:InitiatingHeapOccupancyPercent=25 -XX:GCTimeRatio=25 -XX:CompressedClassSpaceSize=25M -XX:ReservedCodeCacheSize=60M -Xms1100m -Xmx1100m -XX:MaxMetaspaceSize=125M NODE_IP: (v1:status.hostIP) POD_NAME: gooddata-cn-result-cache-7f6bdbcf6f-9hphl (v1:metadata.name) NAMESPACE: gooddata-cn (v1:metadata.namespace) LOGGING_APPENDER: json SPRING_MAIN_BANNER_MODE: off SPRING_CONFIG_ADDITIONAL_LOCATION: classpath:git.properties SPRING_ZIPKIN_ENABLED: false ZIPKIN_HOST: jaeger-collector.monitoring ZIPKIN_PORT: 9411 SPRING_LIFECYCLE_TIMEOUT_PER_SHUTDOWN_PHASE: 60s SPRING_REDIS_CLUSTER_NODES: gooddata-cn-redis-0001-001.w4lhqb.0001.usw2.cache.amazonaws.com:6379,gooddata-cn-redis-0001-002.w4lhqb.0001.usw2.cache.amazonaws.com:6379,gooddata-cn-redis-0002-001.w4lhqb.0001.usw2.cache.amazonaws.com:6379,gooddata-cn-redis-0002-002.w4lhqb.0001.usw2.cache.amazonaws.com:6379 PULSAR_SERVICEURL: pulsar://pulsar-broker.pulsar:6650 PULSAR_ADMINURL: http://pulsar-broker.pulsar:8080 PULSAR_PRODUCERS_RESULT_XTAB_TOPIC: gooddata-cn/gooddata-cn/result.xtab PULSAR_PRODUCERS_RESULT_XTAB_MESSAGE_TTL: 300 PULSAR_CONSUMERS_RESULT_TOPIC: gooddata-cn/gooddata-cn/result.xtab PULSAR_CONSUMERS_RESULT_DEAD_LETTER_TOPIC: gooddata-cn/gooddata-cn/result.xtab.DLQ PULSAR_CONSUMERS_DATA_SOURCE_CHANGE_TOPIC: gooddata-cn/gooddata-cn/data-source.change PULSAR_CONSUMERS_MODEL_UPDATE_TOPIC: gooddata-cn/gooddata-cn/metadata.model PULSAR_CONSUMERS_CACHE_SETTINGS_CHANGE_TOPIC: gooddata-cn/gooddata-cn/cache-settings.change PULSAR_CONSUMERS_CACHE_SETTINGS_CHANGE_MESSAGE_TTL: 300 PULSAR_PRODUCERS_CACHE_SETTINGS_BOOTSTRAP_TOPIC: gooddata-cn/gooddata-cn/cache-settings.bootstrap PULSAR_PRODUCERS_CACHE_SETTINGS_BOOTSTRAP_MESSAGE_TTL: 300 LOG4J_ASYNC_LOGGER_RING_BUFFER_SIZE: 262144 GDC_TELEMETRY_ENABLED: false GDC_TELEMETRY_SITE_ID: 2 LIMIT_MAX_RESULT_XTAB_DIMENSION: 10000 LIMIT_MAX_RESULT_XTAB_CELLS: 1000000 LIMIT_MAX_RESULT_COLUMNS: 1000 GRPC_SERVER_MAX_CONNECTION_AGE: 300 GRPC_SERVER_PERMIT_KEEP_ALIVE_TIME: 25 GRPC_SERVER_PERMIT_KEEP_ALIVE_WITHOUT_CALLS: true SPRING_PROFILES_ACTIVE: default GDC_WORKSPACE_BASELINE_CACHE: 0 GDC_TOTAL_CACHE_LIMIT: 34359738368 AWS_STS_REGIONAL_ENDPOINTS: regional AWS_DEFAULT_REGION: us-west-2 AWS_REGION: us-west-2 AWS_ROLE_ARN: arn:aws:iam::337942936446:role/tf-instance-iam-role-gooddata-cn AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token Mounts: /var/run/secrets/eks.amazonaws.com/serviceaccount from aws-iam-token (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: aws-iam-token: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 86400 QoS Class: Guaranteed Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 52m default-scheduler Successfully assigned gooddata-cn/gooddata-cn-result-cache-7f6bdbcf6f-9hphl to ip-10-163-173-89.us-west-2.compute.internal Normal Pulling 52m kubelet Pulling image "artifactory.cwantools.io:5000/gooddata/result-cache:2.5.1" Normal Pulled 52m kubelet Successfully pulled image "artifactory.cwantools.io:5000/gooddata/result-cache:2.5.1" in 4.971550793s (4.971567631s including waiting) Normal Created 50m (x4 over 52m) kubelet Created container result-cache Normal Started 50m (x4 over 52m) kubelet Started container result-cache Normal Pulled 50m (x3 over 52m) kubelet Container image "artifactory.cwantools.io:5000/gooddata/result-cache:2.5.1" already present on machine Warning Unhealthy 37m (x4 over 52m) kubelet Startup probe failed: Get "http://10.163.190.106:9041/actuator/health/liveness": dial tcp 10.163.190.106:9041: connect: connection refused Warning BackOff 2m55s (x215 over 51m) kubelet Back-off restarting failed container Name: gooddata-cn-result-cache-7f6bdbcf6f-pzkcx Namespace: gooddata-cn Priority: 0 Node: ip-10-163-235-231.us-west-2.compute.internal/10.163.235.231 Start Time: Wed, 01 Nov 2023 16:37:07 +0000 Labels: app.kubernetes.io/component=resultCache app.kubernetes.io/instance=gooddata-cn app.kubernetes.io/name=gooddata-cn pod-template-hash=7f6bdbcf6f Annotations: prometheus.io/path: /actuator/prometheus prometheus.io/port: 9041 prometheus.io/scrape: true Status: Running IP: 10.163.227.240 IPs: IP: 10.163.227.240 Controlled By: ReplicaSet/gooddata-cn-result-cache-7f6bdbcf6f Containers: result-cache: Container ID: containerd://c23b03957b2c924e20933ac7cab6ca5ff3e7030442082d4b1a44d715e386f8bf Image: artifactory.cwantools.io:5000/gooddata/result-cache:2.5.1 Image ID: artifactory.cwantools.io:5000/gooddata/result-cache@sha256:e9cea80137c286f91576d78af899294c192474e2a833349fffe568ccb50ef46a Ports: 6567/TCP, 9041/TCP Host Ports: 0/TCP, 0/TCP State: Running Started: Wed, 01 Nov 2023 16:38:05 +0000 Last State: Terminated Reason: Error Exit Code: 1 Started: Wed, 01 Nov 2023 16:37:30 +0000 Finished: Wed, 01 Nov 2023 16:38:04 +0000 Ready: False Restart Count: 1 Limits: cpu: 750m memory: 2Gi Requests: cpu: 750m memory: 2Gi Liveness: http-get http://:9041/actuator/health/liveness delay=30s timeout=5s period=10s #success=1 #failure=5 Readiness: http-get http://:9041/actuator/health/readiness delay=30s timeout=10s period=10s #success=1 #failure=5 Startup: http-get http://:9041/actuator/health/liveness delay=30s timeout=5s period=10s #success=1 #failure=12 Environment: JDK_JAVA_OPTIONS: -XX:+ExitOnOutOfMemoryError -XX:+UseG1GC -Xss384k -XX:+UseStringDeduplication -XX:MinHeapFreeRatio=15 -XX:MaxHeapFreeRatio=25 -XX:AdaptiveSizePolicyWeight=50 -XX:InitiatingHeapOccupancyPercent=25 -XX:GCTimeRatio=25 -XX:CompressedClassSpaceSize=25M -XX:ReservedCodeCacheSize=60M -Xms1100m -Xmx1100m -XX:MaxMetaspaceSize=125M NODE_IP: (v1:status.hostIP) POD_NAME: gooddata-cn-result-cache-7f6bdbcf6f-pzkcx (v1:metadata.name) NAMESPACE: gooddata-cn (v1:metadata.namespace) LOGGING_APPENDER: json SPRING_MAIN_BANNER_MODE: off SPRING_CONFIG_ADDITIONAL_LOCATION: classpath:git.properties SPRING_ZIPKIN_ENABLED: false ZIPKIN_HOST: jaeger-collector.monitoring ZIPKIN_PORT: 9411 SPRING_LIFECYCLE_TIMEOUT_PER_SHUTDOWN_PHASE: 60s SPRING_REDIS_CLUSTER_NODES: gooddata-cn-redis-0001-001.w4lhqb.0001.usw2.cache.amazonaws.com:6379,gooddata-cn-redis-0001-002.w4lhqb.0001.usw2.cache.amazonaws.com:6379,gooddata-cn-redis-0002-001.w4lhqb.0001.usw2.cache.amazonaws.com:6379,gooddata-cn-redis-0002-002.w4lhqb.0001.usw2.cache.amazonaws.com:6379 PULSAR_SERVICEURL: pulsar://pulsar-broker.pulsar:6650 PULSAR_ADMINURL: http://pulsar-broker.pulsar:8080 PULSAR_PRODUCERS_RESULT_XTAB_TOPIC: gooddata-cn/gooddata-cn/result.xtab PULSAR_PRODUCERS_RESULT_XTAB_MESSAGE_TTL: 300 PULSAR_CONSUMERS_RESULT_TOPIC: gooddata-cn/gooddata-cn/result.xtab PULSAR_CONSUMERS_RESULT_DEAD_LETTER_TOPIC: gooddata-cn/gooddata-cn/result.xtab.DLQ PULSAR_CONSUMERS_DATA_SOURCE_CHANGE_TOPIC: gooddata-cn/gooddata-cn/data-source.change PULSAR_CONSUMERS_MODEL_UPDATE_TOPIC: gooddata-cn/gooddata-cn/metadata.model PULSAR_CONSUMERS_CACHE_SETTINGS_CHANGE_TOPIC: gooddata-cn/gooddata-cn/cache-settings.change PULSAR_CONSUMERS_CACHE_SETTINGS_CHANGE_MESSAGE_TTL: 300 PULSAR_PRODUCERS_CACHE_SETTINGS_BOOTSTRAP_TOPIC: gooddata-cn/gooddata-cn/cache-settings.bootstrap PULSAR_PRODUCERS_CACHE_SETTINGS_BOOTSTRAP_MESSAGE_TTL: 300 LOG4J_ASYNC_LOGGER_RING_BUFFER_SIZE: 262144 GDC_TELEMETRY_ENABLED: false GDC_TELEMETRY_SITE_ID: 2 LIMIT_MAX_RESULT_XTAB_DIMENSION: 10000 LIMIT_MAX_RESULT_XTAB_CELLS: 1000000 LIMIT_MAX_RESULT_COLUMNS: 1000 GRPC_SERVER_MAX_CONNECTION_AGE: 300 GRPC_SERVER_PERMIT_KEEP_ALIVE_TIME: 25 GRPC_SERVER_PERMIT_KEEP_ALIVE_WITHOUT_CALLS: true SPRING_PROFILES_ACTIVE: default GDC_WORKSPACE_BASELINE_CACHE: 0 GDC_TOTAL_CACHE_LIMIT: 34359738368 AWS_STS_REGIONAL_ENDPOINTS: regional AWS_DEFAULT_REGION: us-west-2 AWS_REGION: us-west-2 AWS_ROLE_ARN: arn:aws:iam::337942936446:role/tf-instance-iam-role-gooddata-cn AWS_WEB_IDENTITY_TOKEN_FILE: /var/run/secrets/eks.amazonaws.com/serviceaccount/token Mounts: /var/run/secrets/eks.amazonaws.com/serviceaccount from aws-iam-token (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: aws-iam-token: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 86400 QoS Class: Guaranteed Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 76s default-scheduler Successfully assigned gooddata-cn/gooddata-cn-result-cache-7f6bdbcf6f-pzkcx to ip-10-163-235-231.us-west-2.compute.internal Normal Pulling 74s kubelet Pulling image "artifactory.cwantools.io:5000/gooddata/result-cache:2.5.1" Normal Pulled 53s kubelet Successfully pulled image "artifactory.cwantools.io:5000/gooddata/result-cache:2.5.1" in 20.834430615s (20.834445839s including waiting) Normal Created 18s (x2 over 53s) kubelet Created container result-cache Normal Started 18s (x2 over 53s) kubelet Started container result-cache Normal Pulled 18s kubelet Container image "artifactory.cwantools.io:5000/gooddata/result-cache:2.5.1" already present on machine >