2023-01-25 16:15:24,349 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Received JobGraph submission 'Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15' (39c8a37ddf1096dd2da68216ade26056). 2023-01-25 16:15:24,408 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Submitting job 'Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15' (39c8a37ddf1096dd2da68216ade26056). 2023-01-25 16:15:24,737 INFO org.apache.flink.runtime.jobmanager.DefaultJobGraphStore [] - Added JobGraph(jobId: 39c8a37ddf1096dd2da68216ade26056) to KubernetesStateHandleStore{configMapName='abp-optic-staging-us-east-1-21-cluster-config-map'}. 2023-01-25 16:15:24,737 INFO org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService [] - Starting DefaultLeaderElectionService with org.apache.flink.runtime.leaderelection.MultipleComponentLeaderElectionDriverAdapter@18f5d632. 2023-01-25 16:15:24,846 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at akka://flink/user/rpc/jobmanager_24 . 2023-01-25 16:15:24,846 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Initializing job 'Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15' (39c8a37ddf1096dd2da68216ade26056). 2023-01-25 16:15:24,891 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Using restart back off time strategy FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=20, backoffTimeMS=15000) for Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15 (39c8a37ddf1096dd2da68216ade26056). 2023-01-25 16:15:24,910 INFO org.apache.flink.runtime.checkpoint.DefaultCompletedCheckpointStoreUtils [] - Recovering checkpoints from KubernetesStateHandleStore{configMapName='abp-optic-staging-us-east-1-21-39c8a37ddf1096dd2da68216ade26056-config-map'}. 2023-01-25 16:15:24,914 INFO org.apache.flink.runtime.checkpoint.DefaultCompletedCheckpointStoreUtils [] - Found 0 checkpoints in KubernetesStateHandleStore{configMapName='abp-optic-staging-us-east-1-21-39c8a37ddf1096dd2da68216ade26056-config-map'}. 2023-01-25 16:15:24,914 INFO org.apache.flink.runtime.checkpoint.DefaultCompletedCheckpointStoreUtils [] - Trying to fetch 0 checkpoints from storage. 2023-01-25 16:15:24,926 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Running initialization on master for job Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15 (39c8a37ddf1096dd2da68216ade26056). 2023-01-25 16:15:24,927 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Successfully ran initialization on master in 0 ms. 2023-01-25 16:15:24,968 INFO org.apache.flink.runtime.scheduler.adapter.DefaultExecutionTopology [] - Built 3 new pipelined regions in 0 ms, total 3 pipelined regions currently. 2023-01-25 16:15:24,969 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Using application-defined state backend: org.apache.flink.streaming.api.operators.sorted.state.BatchExecutionStateBackend@4e731cf 2023-01-25 16:15:24,969 INFO org.apache.flink.runtime.state.StateBackendLoader [] - State backend loader loads the state backend as BatchExecutionStateBackend 2023-01-25 16:15:24,969 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Using application defined checkpoint storage: org.apache.flink.streaming.api.operators.sorted.state.BatchExecutionCheckpointStorage@21dbe029 2023-01-25 16:15:24,970 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - No checkpoint found during restore. 2023-01-25 16:15:24,970 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Using failover strategy org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy@daa3037 for Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15 (39c8a37ddf1096dd2da68216ade26056). 2023-01-25 16:15:24,971 INFO org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService [] - Starting DefaultLeaderRetrievalService with KubernetesLeaderRetrievalDriver{configMapName='abp-optic-staging-us-east-1-21-cluster-config-map'}. 2023-01-25 16:15:24,971 INFO org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapSharedInformer [] - Starting to watch for optic/abp-optic-staging-us-east-1-21-cluster-config-map, watching id:74705ef2-2f80-4089-8fd6-7cb2b9e18d49 2023-01-25 16:15:24,971 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Starting execution of job 'Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15' (39c8a37ddf1096dd2da68216ade26056) under job master id 8c239e766e7f3ef40b94f31aa2d741c9. 2023-01-25 16:15:24,971 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Starting split enumerator for source Source: sourceTable[1]. 2023-01-25 16:15:25,088 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Starting scheduling with scheduling strategy [org.apache.flink.runtime.scheduler.strategy.PipelinedRegionSchedulingStrategy] 2023-01-25 16:15:25,088 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15 (39c8a37ddf1096dd2da68216ade26056) switched from state CREATED to RUNNING. 2023-01-25 16:15:25,088 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (2f4ed4e4a4aece0fbd9c5ba3f3c20d46) switched from CREATED to SCHEDULED. 2023-01-25 16:15:25,135 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Connecting to ResourceManager akka.tcp://flink@10.50.91.4:6123/user/rpc/resourcemanager_0(9bf12d9a32b994c6d9f5a26ceac14e70) 2023-01-25 16:15:25,135 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Resolved ResourceManager address, beginning registration 2023-01-25 16:15:25,136 INFO org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService [] - Starting DefaultLeaderRetrievalService with KubernetesLeaderRetrievalDriver{configMapName='abp-optic-staging-us-east-1-21-cluster-config-map'}. 2023-01-25 16:15:25,136 INFO org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMapSharedInformer [] - Starting to watch for optic/abp-optic-staging-us-east-1-21-cluster-config-map, watching id:02fbab6d-a208-419f-b12f-d23b49490aaf 2023-01-25 16:15:25,136 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registering job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:25,250 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registering job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:25,424 INFO org.apache.flink.metrics.datadog.DatadogHttpReporter [] - The metric flink.jobmanager.job.lastCheckpointExternalPath will not be reported because only number types are supported by this reporter. 2023-01-25 16:15:25,469 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registering job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:25,889 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registering job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:26,710 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registering job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:27,824 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registered job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:27,824 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registered job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:27,824 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registered job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:27,824 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registered job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:27,824 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registered job manager 8c239e766e7f3ef40b94f31aa2d741c9@akka.tcp://flink@10.50.91.4:6123/user/rpc/jobmanager_24 for job 39c8a37ddf1096dd2da68216ade26056. 2023-01-25 16:15:27,824 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - JobManager successfully registered at ResourceManager, leader id: 9bf12d9a32b994c6d9f5a26ceac14e70. 2023-01-25 16:15:27,825 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 39c8a37ddf1096dd2da68216ade26056: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}] 2023-01-25 16:15:32,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (2f4ed4e4a4aece0fbd9c5ba3f3c20d46) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:15:32,881 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (attempt #0) with attempt id 2f4ed4e4a4aece0fbd9c5ba3f3c20d46 and vertex id cbc357ccb763df2852fee8c4fc7d55f2_0 to 10.50.87.132:6122-e21296 @ 10-50-87-132.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=37369) with allocation id 2064e9b39de35e4df61b185a9b07c23e 2023-01-25 16:15:37,502 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (2f4ed4e4a4aece0fbd9c5ba3f3c20d46) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:15:37,573 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Source Source: sourceTable[1] registering reader for parallel task 0 @ 10.50.87.132 2023-01-25 16:15:37,574 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Source Source: sourceTable[1] received split request from parallel task 0 2023-01-25 16:15:37,574 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (2f4ed4e4a4aece0fbd9c5ba3f3c20d46) switched from INITIALIZING to RUNNING. 2023-01-25 16:15:37,574 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Subtask 0 (on host '10.50.87.132') is requesting a file source split 2023-01-25 16:15:37,574 INFO org.apache.flink.connector.file.src.assigners.LocalityAwareSplitAssigner [] - Assigning remote split to requesting host '10': Optional[FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-24-22-d4f2cbbd-79bb-4600-80a5-896241e35856.parquet [0, 109648) hosts=[localhost] ID=0000000002 position=null] 2023-01-25 16:15:37,575 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Assigned split to subtask 0 : FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-24-22-d4f2cbbd-79bb-4600-80a5-896241e35856.parquet [0, 109648) hosts=[localhost] ID=0000000002 position=null 2023-01-25 16:15:43,393 WARN akka.remote.ReliableDeliverySupervisor [] - Association with remote system [akka.tcp://flink-metrics@10.50.87.132:37249] has failed, address is now gated for [50] ms. Reason: [Disassociated] 2023-01-25 16:15:43,393 WARN akka.remote.ReliableDeliverySupervisor [] - Association with remote system [akka.tcp://flink-metrics@10.50.87.132:37249] has failed, address is now gated for [50] ms. Reason: [Disassociated] 2023-01-25 16:15:43,394 WARN akka.remote.ReliableDeliverySupervisor [] - Association with remote system [akka.tcp://flink-metrics@10.50.87.132:37249] has failed, address is now gated for [50] ms. Reason: [Disassociated] 2023-01-25 16:15:43,394 WARN akka.remote.ReliableDeliverySupervisor [] - Association with remote system [akka.tcp://flink@10.50.87.132:6122] has failed, address is now gated for [50] ms. Reason: [Disassociated] 2023-01-25 16:15:44,242 WARN akka.remote.transport.netty.NettyTransport [] - Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.50.87.132:6122 2023-01-25 16:15:44,242 WARN akka.remote.ReliableDeliverySupervisor [] - Association with remote system [akka.tcp://flink@10.50.87.132:6122] has failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@10.50.87.132:6122]] Caused by: [java.net.ConnectException: Connection refused: /10.50.87.132:6122] 2023-01-25 16:15:45,123 WARN akka.remote.transport.netty.NettyTransport [] - Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.50.87.132:6122 2023-01-25 16:15:45,123 WARN akka.remote.ReliableDeliverySupervisor [] - Association with remote system [akka.tcp://flink@10.50.87.132:6122] has failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@10.50.87.132:6122]] Caused by: [java.net.ConnectException: Connection refused: /10.50.87.132:6122] 2023-01-25 16:15:46,483 WARN akka.remote.transport.netty.NettyTransport [] - Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.50.87.132:6122 2023-01-25 16:15:46,484 WARN akka.remote.ReliableDeliverySupervisor [] - Association with remote system [akka.tcp://flink@10.50.87.132:6122] has failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@10.50.87.132:6122]] Caused by: [java.net.ConnectException: Connection refused: /10.50.87.132:6122] 2023-01-25 16:15:51,942 WARN akka.remote.transport.netty.NettyTransport [] - Remote connection to [null] failed with java.net.ConnectException: Connection refused: /10.50.87.132:6122 2023-01-25 16:15:51,942 WARN akka.remote.ReliableDeliverySupervisor [] - Association with remote system [akka.tcp://flink@10.50.87.132:6122] has failed, address is now gated for [50] ms. Reason: [Association failed with [akka.tcp://flink@10.50.87.132:6122]] Caused by: [java.net.ConnectException: Connection refused: /10.50.87.132:6122] 2023-01-25 16:15:54,464 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - TaskManager with id 10.50.87.132:6122-e21296 is no longer reachable. 2023-01-25 16:15:54,466 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (25d8b821b14861f825d21418b4f5dfff) switched from RUNNING to FAILED on 10.50.87.132:6122-e21296 @ 10-50-87-132.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=37369). org.apache.flink.runtime.jobmaster.JobMasterException: TaskManager with id 10.50.87.132:6122-e21296 is no longer reachable. at org.apache.flink.runtime.jobmaster.JobMaster$TaskManagerHeartbeatListener.notifyTargetUnreachable(JobMaster.java:1387) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatMonitorImpl.reportHeartbeatRpcFailure(HeartbeatMonitorImpl.java:123) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.runIfHeartbeatMonitorExists(HeartbeatManagerImpl.java:275) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.reportHeartbeatTargetUnreachable(HeartbeatManagerImpl.java:267) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.handleHeartbeatRpcFailure(HeartbeatManagerImpl.java:262) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.lambda$handleHeartbeatRpc$0(HeartbeatManagerImpl.java:248) ~[flink-dist-1.15.2.jar:1.15.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$Completion.run(Unknown Source) ~[?:?] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRunAsync$4(AkkaRpcActor.java:443) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:443) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:213) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.ActorCell.invoke(ActorCell.scala:548) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?] at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?] 2023-01-25 16:15:54,467 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 25d8b821b14861f825d21418b4f5dfff. 2023-01-25 16:15:54,467 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 25d8b821b14861f825d21418b4f5dfff. 2023-01-25 16:15:54,467 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - Calculating tasks to restart to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_4. 2023-01-25 16:15:54,467 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - 15 tasks should be restarted to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_4. 2023-01-25 16:15:54,467 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Bon com.imperva.combine.BillingAggregateJob (fab0c3a6fdc1c9d4271724d0e48bb1d9) switched from state RUNNING to RESTARTING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (5bfff273ad67fea0a4293159306a53c3) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (263b6bc44f7983468c3178734612764e) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (84de435dcc64df44fc5f2aed0ab94ab8) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (806472748402a2e6a87cf658b89c7e33) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (b7a8687d73363e2d8f8a9ff5b52493c6) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (b2d0051181873ee446326b33b8ac2284) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (44f4cc187dc7dbf5744fe4763fc04300) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (e62b4f1730fc550cbb5f5db3ebf6de6f) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (92f8dd1264ef5beab0c18fa179255098) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (f217f98a1db54836ea84e6bc0c622891) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (c807bab30e61d018a19d0a69cc681df4) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (41f33b228549cd0b6f8f44dc4a02bad4) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (82b555b6bf9913f40fd17b94ba74c3ef) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (aa01bbb580e53a768801ac28ede07020) switched from RUNNING to CANCELING. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (5bfff273ad67fea0a4293159306a53c3) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 5bfff273ad67fea0a4293159306a53c3. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (41f33b228549cd0b6f8f44dc4a02bad4) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 41f33b228549cd0b6f8f44dc4a02bad4. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 41f33b228549cd0b6f8f44dc4a02bad4. 2023-01-25 16:15:54,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (e62b4f1730fc550cbb5f5db3ebf6de6f) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution e62b4f1730fc550cbb5f5db3ebf6de6f. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution e62b4f1730fc550cbb5f5db3ebf6de6f. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (263b6bc44f7983468c3178734612764e) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 263b6bc44f7983468c3178734612764e. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (82b555b6bf9913f40fd17b94ba74c3ef) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 82b555b6bf9913f40fd17b94ba74c3ef. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 82b555b6bf9913f40fd17b94ba74c3ef. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (92f8dd1264ef5beab0c18fa179255098) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 92f8dd1264ef5beab0c18fa179255098. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 92f8dd1264ef5beab0c18fa179255098. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (84de435dcc64df44fc5f2aed0ab94ab8) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 84de435dcc64df44fc5f2aed0ab94ab8. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (aa01bbb580e53a768801ac28ede07020) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution aa01bbb580e53a768801ac28ede07020. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution aa01bbb580e53a768801ac28ede07020. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (b7a8687d73363e2d8f8a9ff5b52493c6) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution b7a8687d73363e2d8f8a9ff5b52493c6. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (c807bab30e61d018a19d0a69cc681df4) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution c807bab30e61d018a19d0a69cc681df4. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution c807bab30e61d018a19d0a69cc681df4. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (44f4cc187dc7dbf5744fe4763fc04300) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 44f4cc187dc7dbf5744fe4763fc04300. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 44f4cc187dc7dbf5744fe4763fc04300. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (806472748402a2e6a87cf658b89c7e33) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 806472748402a2e6a87cf658b89c7e33. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (f217f98a1db54836ea84e6bc0c622891) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution f217f98a1db54836ea84e6bc0c622891. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution f217f98a1db54836ea84e6bc0c622891. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (b2d0051181873ee446326b33b8ac2284) switched from CANCELING to CANCELED. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution b2d0051181873ee446326b33b8ac2284. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution b2d0051181873ee446326b33b8ac2284. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [195d37bc3fb68c9e107990d2499b16db]. 2023-01-25 16:15:54,471 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [3faf71d64bece5ea7032f6c2bebeca82]. 2023-01-25 16:15:54,472 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [bd93647eee20bb700c17674fc7299f2b]. 2023-01-25 16:15:54,472 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [3534d8bce7f4573ec03d2d6ea3bc0bd5]. 2023-01-25 16:15:54,472 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [7d6d50a223b209984d0e9cf0d43c0c5c]. 2023-01-25 16:15:54,472 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job fab0c3a6fdc1c9d4271724d0e48bb1d9 2023-01-25 16:15:55,144 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - TaskManager with id 10.50.87.132:6122-e21296 is no longer reachable. 2023-01-25 16:15:55,144 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (2f4ed4e4a4aece0fbd9c5ba3f3c20d46) switched from RUNNING to FAILED on 10.50.87.132:6122-e21296 @ 10-50-87-132.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=37369). org.apache.flink.runtime.jobmaster.JobMasterException: TaskManager with id 10.50.87.132:6122-e21296 is no longer reachable. at org.apache.flink.runtime.jobmaster.JobMaster$TaskManagerHeartbeatListener.notifyTargetUnreachable(JobMaster.java:1387) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatMonitorImpl.reportHeartbeatRpcFailure(HeartbeatMonitorImpl.java:123) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.runIfHeartbeatMonitorExists(HeartbeatManagerImpl.java:275) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.reportHeartbeatTargetUnreachable(HeartbeatManagerImpl.java:267) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.handleHeartbeatRpcFailure(HeartbeatManagerImpl.java:262) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.lambda$handleHeartbeatRpc$0(HeartbeatManagerImpl.java:248) ~[flink-dist-1.15.2.jar:1.15.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$Completion.run(Unknown Source) ~[?:?] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRunAsync$4(AkkaRpcActor.java:443) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:443) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:213) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.ActorCell.invoke(ActorCell.scala:548) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?] at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?] 2023-01-25 16:15:55,145 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 2f4ed4e4a4aece0fbd9c5ba3f3c20d46. 2023-01-25 16:15:55,145 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 2f4ed4e4a4aece0fbd9c5ba3f3c20d46. 2023-01-25 16:15:55,146 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - Calculating tasks to restart to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0. 2023-01-25 16:15:55,146 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Removing registered reader after failure for subtask 0 of source Source: sourceTable[1]. 2023-01-25 16:15:55,146 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - 1 tasks should be restarted to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0. 2023-01-25 16:15:55,146 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15 (39c8a37ddf1096dd2da68216ade26056) switched from state RUNNING to RESTARTING. 2023-01-25 16:15:55,146 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [2064e9b39de35e4df61b185a9b07c23e]. 2023-01-25 16:15:55,146 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 39c8a37ddf1096dd2da68216ade26056 2023-01-25 16:15:56,503 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - TaskManager with id 10.50.87.132:6122-e21296 is no longer reachable. 2023-01-25 16:15:56,503 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Closing TaskExecutor connection 10.50.87.132:6122-e21296 because: TaskManager with id 10.50.87.132:6122-e21296 is no longer reachable. 2023-01-25 16:15:57,940 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registering TaskManager with ResourceID 10.50.87.132:6122-1eeec8 (akka.tcp://flink@10.50.87.132:6122/user/rpc/taskmanager_0) at ResourceManager 2023-01-25 16:16:01,963 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - TaskManager with id 10.50.87.132:6122-e21296 is no longer reachable. 2023-01-25 16:16:01,964 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (f72b637e4b71463d0985c55a9ddd69c5) switched from RUNNING to FAILED on 10.50.87.132:6122-e21296 @ 10-50-87-132.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=37369). org.apache.flink.runtime.jobmaster.JobMasterException: TaskManager with id 10.50.87.132:6122-e21296 is no longer reachable. at org.apache.flink.runtime.jobmaster.JobMaster$TaskManagerHeartbeatListener.notifyTargetUnreachable(JobMaster.java:1387) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatMonitorImpl.reportHeartbeatRpcFailure(HeartbeatMonitorImpl.java:123) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.runIfHeartbeatMonitorExists(HeartbeatManagerImpl.java:275) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.reportHeartbeatTargetUnreachable(HeartbeatManagerImpl.java:267) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.handleHeartbeatRpcFailure(HeartbeatManagerImpl.java:262) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.heartbeat.HeartbeatManagerImpl.lambda$handleHeartbeatRpc$0(HeartbeatManagerImpl.java:248) ~[flink-dist-1.15.2.jar:1.15.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$Completion.run(Unknown Source) ~[?:?] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRunAsync$4(AkkaRpcActor.java:443) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:443) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:213) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163) ~[flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.actor.ActorCell.invoke(ActorCell.scala:548) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_cf84c199-b293-40ec-9bc0-79d68a36bad6.jar:1.15.2] at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool.scan(Unknown Source) [?:?] at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) [?:?] at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) [?:?] 2023-01-25 16:16:01,964 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution f72b637e4b71463d0985c55a9ddd69c5. 2023-01-25 16:16:01,965 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - Calculating tasks to restart to recover the failed task a3d766fbaa6452007146a3fb11a8b6de_1. 2023-01-25 16:16:01,965 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - 15 tasks should be restarted to recover the failed task a3d766fbaa6452007146a3fb11a8b6de_1. 2023-01-25 16:16:01,965 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Bon com.imperva.combine.ScraperTrafficAggregationJob (2c99420981b3238c08535d40fbaacc63) switched from state RUNNING to RESTARTING. 2023-01-25 16:16:01,965 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (b78cbad7b1fc02890fc7fa09815ac512) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,965 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (c941a2732bdb2c0e56d8e88257de5229) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,965 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (733f0316ce4a8c3688ae247e04f017a7) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (6a9187faeefcb3d9cdfb03971c98b0fc) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (d5920d9a7e51cb8e5f6e4295b17baa6f) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (8b711385133e18408c3d72a34708bba8) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (370a96a8a7081609bc920f340ec14d87) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (d8be6397e168084d551a3d7275126276) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (c10ec2b045ef9d0a9256cef71cb7dbe1) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (693b427ccbf7f174a1802cd8430a22e5) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (8bd3445ade92cbc07ba06ee5e5b12846) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (90d22f5f5c751e40bca123311a72efa6) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (ed916d78709e27afaeef89cca28319f4) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (8efa96cf94dfc4ce654248d7ef1c2322) switched from RUNNING to CANCELING. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (693b427ccbf7f174a1802cd8430a22e5) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 693b427ccbf7f174a1802cd8430a22e5. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 693b427ccbf7f174a1802cd8430a22e5. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (d5920d9a7e51cb8e5f6e4295b17baa6f) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution d5920d9a7e51cb8e5f6e4295b17baa6f. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution d5920d9a7e51cb8e5f6e4295b17baa6f. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (d8be6397e168084d551a3d7275126276) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution d8be6397e168084d551a3d7275126276. 2023-01-25 16:16:01,967 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution d8be6397e168084d551a3d7275126276. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (c941a2732bdb2c0e56d8e88257de5229) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution c941a2732bdb2c0e56d8e88257de5229. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (ed916d78709e27afaeef89cca28319f4) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution ed916d78709e27afaeef89cca28319f4. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution ed916d78709e27afaeef89cca28319f4. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (c10ec2b045ef9d0a9256cef71cb7dbe1) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution c10ec2b045ef9d0a9256cef71cb7dbe1. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution c10ec2b045ef9d0a9256cef71cb7dbe1. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (733f0316ce4a8c3688ae247e04f017a7) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 733f0316ce4a8c3688ae247e04f017a7. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (8efa96cf94dfc4ce654248d7ef1c2322) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 8efa96cf94dfc4ce654248d7ef1c2322. 2023-01-25 16:16:01,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 8efa96cf94dfc4ce654248d7ef1c2322. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (370a96a8a7081609bc920f340ec14d87) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 370a96a8a7081609bc920f340ec14d87. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 370a96a8a7081609bc920f340ec14d87. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (b78cbad7b1fc02890fc7fa09815ac512) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution b78cbad7b1fc02890fc7fa09815ac512. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (90d22f5f5c751e40bca123311a72efa6) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 90d22f5f5c751e40bca123311a72efa6. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 90d22f5f5c751e40bca123311a72efa6. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (6a9187faeefcb3d9cdfb03971c98b0fc) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 6a9187faeefcb3d9cdfb03971c98b0fc. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (8bd3445ade92cbc07ba06ee5e5b12846) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 8bd3445ade92cbc07ba06ee5e5b12846. 2023-01-25 16:16:01,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 8bd3445ade92cbc07ba06ee5e5b12846. 2023-01-25 16:16:01,970 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (8b711385133e18408c3d72a34708bba8) switched from CANCELING to CANCELED. 2023-01-25 16:16:01,970 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 8b711385133e18408c3d72a34708bba8. 2023-01-25 16:16:01,970 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Discarding the results produced by task execution 8b711385133e18408c3d72a34708bba8. 2023-01-25 16:16:01,970 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [740e02a03b123fee1b5a7cfa6e478e15]. 2023-01-25 16:16:01,970 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [320a6c79f32ac61db11fa2aeadfb259e]. 2023-01-25 16:16:01,970 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [c484d6319df7408fcfeb47dc6f00638e]. 2023-01-25 16:16:01,970 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [53096626a41bb316f72e87cc7fb17a68]. 2023-01-25 16:16:01,970 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [6c63c8d54701507500fdc6376c8b7a8f]. 2023-01-25 16:16:01,970 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 2c99420981b3238c08535d40fbaacc63 2023-01-25 16:16:09,469 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Bon com.imperva.combine.BillingAggregateJob (fab0c3a6fdc1c9d4271724d0e48bb1d9) switched from state RESTARTING to RUNNING. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - No checkpoint found during restore. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (608a5960d2d2604d26a70923d1505a77) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (cc2f2dc9794c09017f8d647f48d513fd) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (a1d3d48fb952a2f63346df4720389299) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (50717497e98366ea810e774f40b811b4) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (70d89ba0990a5c2621a6de578fee17da) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (345e8409703eb2a18577432e63bce5d4) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (e63739b9e1ade17effce29c8c42def11) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (f38bdcbc97a88b1458944a37d82e82b4) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (da9183e3c518f8edc477d4f15ee7cfda) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (0069a35cdf0de2b7fdb7c564096c4f34) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (2f54bda3e247b252321d01378473262a) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (112db576054abe69866e385f8215ef9b) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (1fc97691f59b7274f807c3c0fd9c2091) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (03576a7bf3a01cd3703d86a088dc1b02) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,470 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (71c418811bfb70843ef682b6b257a9e2) switched from CREATED to SCHEDULED. 2023-01-25 16:16:09,471 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job fab0c3a6fdc1c9d4271724d0e48bb1d9: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}] 2023-01-25 16:16:09,471 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job fab0c3a6fdc1c9d4271724d0e48bb1d9: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=2}] 2023-01-25 16:16:09,471 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job fab0c3a6fdc1c9d4271724d0e48bb1d9: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=3}] 2023-01-25 16:16:09,471 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job fab0c3a6fdc1c9d4271724d0e48bb1d9: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=4}] 2023-01-25 16:16:09,471 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job fab0c3a6fdc1c9d4271724d0e48bb1d9: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=5}] 2023-01-25 16:16:10,147 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15 (39c8a37ddf1096dd2da68216ade26056) switched from state RESTARTING to RUNNING. 2023-01-25 16:16:10,147 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - No checkpoint found during restore. 2023-01-25 16:16:10,147 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Recovering subtask 0 to checkpoint -1 for source Source: sourceTable[1] to checkpoint. 2023-01-25 16:16:10,147 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (ecb2168e46b0b185c251b6c198d32a85) switched from CREATED to SCHEDULED. 2023-01-25 16:16:10,147 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 39c8a37ddf1096dd2da68216ade26056: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}] 2023-01-25 16:16:12,877 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (ecb2168e46b0b185c251b6c198d32a85) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,877 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (608a5960d2d2604d26a70923d1505a77) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (attempt #1) with attempt id ecb2168e46b0b185c251b6c198d32a85 and vertex id cbc357ccb763df2852fee8c4fc7d55f2_0 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id aa1faf216922af1a0d471158553e9bc5 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (1/5) (attempt #5) with attempt id 608a5960d2d2604d26a70923d1505a77 and vertex id cbc357ccb763df2852fee8c4fc7d55f2_0 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 5ad1fc63fb7f8c6745febb7409ad0b4b 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (cc2f2dc9794c09017f8d647f48d513fd) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (2/5) (attempt #5) with attempt id cc2f2dc9794c09017f8d647f48d513fd and vertex id cbc357ccb763df2852fee8c4fc7d55f2_1 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id d86788f85b450aa5ed85cd0c027707bf 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (a1d3d48fb952a2f63346df4720389299) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (3/5) (attempt #5) with attempt id a1d3d48fb952a2f63346df4720389299 and vertex id cbc357ccb763df2852fee8c4fc7d55f2_2 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id a7539fb3d2a2101bf5596a12e96d32a1 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (50717497e98366ea810e774f40b811b4) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (4/5) (attempt #5) with attempt id 50717497e98366ea810e774f40b811b4 and vertex id cbc357ccb763df2852fee8c4fc7d55f2_3 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 08adb180fc83b52d954e9df6597c5854 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (70d89ba0990a5c2621a6de578fee17da) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (5/5) (attempt #5) with attempt id 70d89ba0990a5c2621a6de578fee17da and vertex id cbc357ccb763df2852fee8c4fc7d55f2_4 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id a4b391cb27f04f5b6836c33bd78178f5 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (345e8409703eb2a18577432e63bce5d4) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,879 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (attempt #5) with attempt id 345e8409703eb2a18577432e63bce5d4 and vertex id d28463815a9d818025b1ff96211a6dc9_0 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 5ad1fc63fb7f8c6745febb7409ad0b4b 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (e63739b9e1ade17effce29c8c42def11) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (attempt #5) with attempt id e63739b9e1ade17effce29c8c42def11 and vertex id d28463815a9d818025b1ff96211a6dc9_1 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id d86788f85b450aa5ed85cd0c027707bf 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (f38bdcbc97a88b1458944a37d82e82b4) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (attempt #5) with attempt id f38bdcbc97a88b1458944a37d82e82b4 and vertex id d28463815a9d818025b1ff96211a6dc9_2 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id a7539fb3d2a2101bf5596a12e96d32a1 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (da9183e3c518f8edc477d4f15ee7cfda) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (attempt #5) with attempt id da9183e3c518f8edc477d4f15ee7cfda and vertex id d28463815a9d818025b1ff96211a6dc9_3 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 08adb180fc83b52d954e9df6597c5854 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (0069a35cdf0de2b7fdb7c564096c4f34) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (attempt #5) with attempt id 0069a35cdf0de2b7fdb7c564096c4f34 and vertex id d28463815a9d818025b1ff96211a6dc9_4 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id a4b391cb27f04f5b6836c33bd78178f5 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (2f54bda3e247b252321d01378473262a) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,880 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (attempt #5) with attempt id 2f54bda3e247b252321d01378473262a and vertex id a3d766fbaa6452007146a3fb11a8b6de_0 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 5ad1fc63fb7f8c6745febb7409ad0b4b 2023-01-25 16:16:12,881 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (112db576054abe69866e385f8215ef9b) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,881 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (attempt #5) with attempt id 112db576054abe69866e385f8215ef9b and vertex id a3d766fbaa6452007146a3fb11a8b6de_1 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id d86788f85b450aa5ed85cd0c027707bf 2023-01-25 16:16:12,881 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (1fc97691f59b7274f807c3c0fd9c2091) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,881 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (attempt #5) with attempt id 1fc97691f59b7274f807c3c0fd9c2091 and vertex id a3d766fbaa6452007146a3fb11a8b6de_2 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id a7539fb3d2a2101bf5596a12e96d32a1 2023-01-25 16:16:12,881 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (03576a7bf3a01cd3703d86a088dc1b02) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,881 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (attempt #5) with attempt id 03576a7bf3a01cd3703d86a088dc1b02 and vertex id a3d766fbaa6452007146a3fb11a8b6de_3 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 08adb180fc83b52d954e9df6597c5854 2023-01-25 16:16:12,881 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (71c418811bfb70843ef682b6b257a9e2) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:12,881 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (attempt #5) with attempt id 71c418811bfb70843ef682b6b257a9e2 and vertex id a3d766fbaa6452007146a3fb11a8b6de_4 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id a4b391cb27f04f5b6836c33bd78178f5 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Bon com.imperva.combine.ScraperTrafficAggregationJob (2c99420981b3238c08535d40fbaacc63) switched from state RESTARTING to RUNNING. 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - No checkpoint found during restore. 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (dcfeafd0a7117b25026d78e1d225bf7e) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (98b7482d84499b49fa50af0a33583b33) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (a0a012a1a3d97b7d2886090fa85285fe) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (e62016ca685ebcca7325dae16a154564) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (138e52c1de039ca62d0cfe877fcb234c) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (56e93b5b51a10f381470ec5bde29ddd9) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (dd2b0692578f11730568c7af7503a2bb) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,968 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (bffffce9a7a90235f81ee0aa59f2a328) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (1ba199de6b59d6d3d33e182b33ae2925) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (6e7deb57084d79841bf66b7863f3e4a2) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (4ad0709b347b6274c61b6f4baab17463) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (cd8ba03f3ca4b91154c1a263fe2a6452) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (72ebd2c357c8f87d889ba0fb1f58ed5c) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (dce840afd2fc123bb1e76961fc0aa400) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,969 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (f5fc8a0778dcdc57ce14318f72ea5f2a) switched from CREATED to SCHEDULED. 2023-01-25 16:16:16,969 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 2c99420981b3238c08535d40fbaacc63: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}] 2023-01-25 16:16:16,970 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 2c99420981b3238c08535d40fbaacc63: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=2}] 2023-01-25 16:16:16,970 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 2c99420981b3238c08535d40fbaacc63: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=3}] 2023-01-25 16:16:16,970 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 2c99420981b3238c08535d40fbaacc63: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=4}] 2023-01-25 16:16:16,970 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 2c99420981b3238c08535d40fbaacc63: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=5}] 2023-01-25 16:16:16,971 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (dcfeafd0a7117b25026d78e1d225bf7e) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (1/5) (attempt #6) with attempt id dcfeafd0a7117b25026d78e1d225bf7e and vertex id cbc357ccb763df2852fee8c4fc7d55f2_0 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id a705bbda65ad096f01d847f3f6bb69e2 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (98b7482d84499b49fa50af0a33583b33) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (2/5) (attempt #6) with attempt id 98b7482d84499b49fa50af0a33583b33 and vertex id cbc357ccb763df2852fee8c4fc7d55f2_1 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 8595466b18c410257d8dd95850930c49 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (a0a012a1a3d97b7d2886090fa85285fe) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (3/5) (attempt #6) with attempt id a0a012a1a3d97b7d2886090fa85285fe and vertex id cbc357ccb763df2852fee8c4fc7d55f2_2 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 6bb4de4bb9119ff0fcb9d1e733681dfa 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (e62016ca685ebcca7325dae16a154564) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (4/5) (attempt #6) with attempt id e62016ca685ebcca7325dae16a154564 and vertex id cbc357ccb763df2852fee8c4fc7d55f2_3 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id ccbbe84bc3d2eed4729ddddd8aac07cb 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (138e52c1de039ca62d0cfe877fcb234c) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: Custom Source -> Filter -> Flat Map (5/5) (attempt #6) with attempt id 138e52c1de039ca62d0cfe877fcb234c and vertex id cbc357ccb763df2852fee8c4fc7d55f2_4 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 3b88916e604d7251132b77a302618166 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (56e93b5b51a10f381470ec5bde29ddd9) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,972 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (attempt #6) with attempt id 56e93b5b51a10f381470ec5bde29ddd9 and vertex id d28463815a9d818025b1ff96211a6dc9_0 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id a705bbda65ad096f01d847f3f6bb69e2 2023-01-25 16:16:16,973 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (dd2b0692578f11730568c7af7503a2bb) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,973 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (attempt #6) with attempt id dd2b0692578f11730568c7af7503a2bb and vertex id d28463815a9d818025b1ff96211a6dc9_1 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 8595466b18c410257d8dd95850930c49 2023-01-25 16:16:16,973 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (bffffce9a7a90235f81ee0aa59f2a328) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,973 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (attempt #6) with attempt id bffffce9a7a90235f81ee0aa59f2a328 and vertex id d28463815a9d818025b1ff96211a6dc9_2 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 6bb4de4bb9119ff0fcb9d1e733681dfa 2023-01-25 16:16:16,973 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (1ba199de6b59d6d3d33e182b33ae2925) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,973 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (attempt #6) with attempt id 1ba199de6b59d6d3d33e182b33ae2925 and vertex id d28463815a9d818025b1ff96211a6dc9_3 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id ccbbe84bc3d2eed4729ddddd8aac07cb 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (6e7deb57084d79841bf66b7863f3e4a2) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (attempt #6) with attempt id 6e7deb57084d79841bf66b7863f3e4a2 and vertex id d28463815a9d818025b1ff96211a6dc9_4 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 3b88916e604d7251132b77a302618166 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (4ad0709b347b6274c61b6f4baab17463) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (attempt #6) with attempt id 4ad0709b347b6274c61b6f4baab17463 and vertex id a3d766fbaa6452007146a3fb11a8b6de_0 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id a705bbda65ad096f01d847f3f6bb69e2 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (cd8ba03f3ca4b91154c1a263fe2a6452) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (attempt #6) with attempt id cd8ba03f3ca4b91154c1a263fe2a6452 and vertex id a3d766fbaa6452007146a3fb11a8b6de_1 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 8595466b18c410257d8dd95850930c49 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (72ebd2c357c8f87d889ba0fb1f58ed5c) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (attempt #6) with attempt id 72ebd2c357c8f87d889ba0fb1f58ed5c and vertex id a3d766fbaa6452007146a3fb11a8b6de_2 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 6bb4de4bb9119ff0fcb9d1e733681dfa 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (dce840afd2fc123bb1e76961fc0aa400) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (attempt #6) with attempt id dce840afd2fc123bb1e76961fc0aa400 and vertex id a3d766fbaa6452007146a3fb11a8b6de_3 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id ccbbe84bc3d2eed4729ddddd8aac07cb 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (f5fc8a0778dcdc57ce14318f72ea5f2a) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:16,974 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (attempt #6) with attempt id f5fc8a0778dcdc57ce14318f72ea5f2a and vertex id a3d766fbaa6452007146a3fb11a8b6de_4 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id 3b88916e604d7251132b77a302618166 2023-01-25 16:16:17,873 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (608a5960d2d2604d26a70923d1505a77) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:18,393 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (608a5960d2d2604d26a70923d1505a77) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:21,442 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (03576a7bf3a01cd3703d86a088dc1b02) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,443 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (71c418811bfb70843ef682b6b257a9e2) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,445 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (2f54bda3e247b252321d01378473262a) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,450 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (112db576054abe69866e385f8215ef9b) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,450 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (1fc97691f59b7274f807c3c0fd9c2091) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,466 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (f38bdcbc97a88b1458944a37d82e82b4) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,466 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (0069a35cdf0de2b7fdb7c564096c4f34) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,481 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (345e8409703eb2a18577432e63bce5d4) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,486 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (da9183e3c518f8edc477d4f15ee7cfda) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,511 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (a1d3d48fb952a2f63346df4720389299) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,527 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (cc2f2dc9794c09017f8d647f48d513fd) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,559 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (50717497e98366ea810e774f40b811b4) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,560 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (70d89ba0990a5c2621a6de578fee17da) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,591 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (a1d3d48fb952a2f63346df4720389299) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:21,592 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (cc2f2dc9794c09017f8d647f48d513fd) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:21,600 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (e63739b9e1ade17effce29c8c42def11) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,629 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (50717497e98366ea810e774f40b811b4) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:21,668 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (70d89ba0990a5c2621a6de578fee17da) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:21,732 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (4ad0709b347b6274c61b6f4baab17463) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,755 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (72ebd2c357c8f87d889ba0fb1f58ed5c) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,755 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (f5fc8a0778dcdc57ce14318f72ea5f2a) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,755 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (dce840afd2fc123bb1e76961fc0aa400) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,755 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (cd8ba03f3ca4b91154c1a263fe2a6452) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,755 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (1ba199de6b59d6d3d33e182b33ae2925) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,761 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (a0a012a1a3d97b7d2886090fa85285fe) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,779 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (98b7482d84499b49fa50af0a33583b33) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,784 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (6e7deb57084d79841bf66b7863f3e4a2) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,786 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (bffffce9a7a90235f81ee0aa59f2a328) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,791 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (138e52c1de039ca62d0cfe877fcb234c) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,791 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (dcfeafd0a7117b25026d78e1d225bf7e) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,791 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (e62016ca685ebcca7325dae16a154564) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,799 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (dd2b0692578f11730568c7af7503a2bb) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:21,810 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (56e93b5b51a10f381470ec5bde29ddd9) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:22,209 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (da9183e3c518f8edc477d4f15ee7cfda) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,220 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (2f54bda3e247b252321d01378473262a) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,223 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (112db576054abe69866e385f8215ef9b) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,228 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (03576a7bf3a01cd3703d86a088dc1b02) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,228 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (1fc97691f59b7274f807c3c0fd9c2091) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,243 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (71c418811bfb70843ef682b6b257a9e2) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,245 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (345e8409703eb2a18577432e63bce5d4) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,247 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (e63739b9e1ade17effce29c8c42def11) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,257 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (0069a35cdf0de2b7fdb7c564096c4f34) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,278 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (f38bdcbc97a88b1458944a37d82e82b4) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,302 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (5/5) (6e7deb57084d79841bf66b7863f3e4a2) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,424 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (3/5) (bffffce9a7a90235f81ee0aa59f2a328) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,426 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (5/5) (f5fc8a0778dcdc57ce14318f72ea5f2a) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,435 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (2/5) (dd2b0692578f11730568c7af7503a2bb) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,439 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (4/5) (dce840afd2fc123bb1e76961fc0aa400) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,450 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (1/5) (56e93b5b51a10f381470ec5bde29ddd9) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,464 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (3/5) (72ebd2c357c8f87d889ba0fb1f58ed5c) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,468 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - TumblingEventTimeWindows -> (Process, Sink: postgres sink) (4/5) (1ba199de6b59d6d3d33e182b33ae2925) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,481 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (2/5) (cd8ba03f3ca4b91154c1a263fe2a6452) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:22,506 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - ProcessingTimeSessionWindows -> Sink: side output sink (1/5) (4ad0709b347b6274c61b6f4baab17463) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:23,057 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (4/5) (e62016ca685ebcca7325dae16a154564) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:23,087 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (3/5) (a0a012a1a3d97b7d2886090fa85285fe) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:23,097 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (2/5) (98b7482d84499b49fa50af0a33583b33) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:23,111 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (1/5) (dcfeafd0a7117b25026d78e1d225bf7e) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:23,118 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Custom Source -> Filter -> Flat Map (5/5) (138e52c1de039ca62d0cfe877fcb234c) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:28,456 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (ecb2168e46b0b185c251b6c198d32a85) switched from DEPLOYING to FAILED on 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835). java.lang.OutOfMemoryError: Can't allocate enough direct buffer for batch shuffle read buffer pool (bytes allocated: 2392064, bytes still needed: 64716800). To avoid the exception, you need to do one of the following adjustments: 1) If you have ever decreased taskmanager.memory.framework.off-heap.size, you need to undo the decrement; 2) If you ever increased taskmanager.memory.framework.off-heap.batch-shuffle.size, you should also increase taskmanager.memory.framework.off-heap.size; 3) If neither the above cases, it usually means some other parts of your application have consumed too many direct memory and the value of taskmanager.memory.task.off-heap.size should be increased. at org.apache.flink.runtime.io.disk.BatchShuffleReadBufferPool.initialize(BatchShuffleReadBufferPool.java:164) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.io.network.partition.SortMergeResultPartition.setup(SortMergeResultPartition.java:189) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.taskmanager.Task.setupPartitionsAndGates(Task.java:959) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:652) ~[flink-dist-1.15.2.jar:1.15.2] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) ~[flink-dist-1.15.2.jar:1.15.2] at java.lang.Thread.run(Unknown Source) ~[?:?] 2023-01-25 16:16:28,462 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 39c8a37ddf1096dd2da68216ade26056 2023-01-25 16:16:28,462 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - Calculating tasks to restart to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0. 2023-01-25 16:16:28,462 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Removing registered reader after failure for subtask 0 of source Source: sourceTable[1]. 2023-01-25 16:16:28,463 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - 1 tasks should be restarted to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0. 2023-01-25 16:16:28,463 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15 (39c8a37ddf1096dd2da68216ade26056) switched from state RUNNING to RESTARTING. 2023-01-25 16:16:43,463 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Stats Engine Usage Report 2023-01-25T15 - 2023-01-25T15 (39c8a37ddf1096dd2da68216ade26056) switched from state RESTARTING to RUNNING. 2023-01-25 16:16:43,464 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - No checkpoint found during restore. 2023-01-25 16:16:43,464 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Recovering subtask 0 to checkpoint -1 for source Source: sourceTable[1] to checkpoint. 2023-01-25 16:16:43,464 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (a2428909648e0321ca1133dd233cd9b4) switched from CREATED to SCHEDULED. 2023-01-25 16:16:43,465 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 39c8a37ddf1096dd2da68216ade26056: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}] 2023-01-25 16:16:43,465 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (a2428909648e0321ca1133dd233cd9b4) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:43,465 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (attempt #2) with attempt id a2428909648e0321ca1133dd233cd9b4 and vertex id cbc357ccb763df2852fee8c4fc7d55f2_0 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id aa1faf216922af1a0d471158553e9bc5 2023-01-25 16:16:43,480 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (a2428909648e0321ca1133dd233cd9b4) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:43,903 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Source Source: sourceTable[1] registering reader for parallel task 0 @ 10.50.89.193 2023-01-25 16:16:43,904 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Source Source: sourceTable[1] received split request from parallel task 0 2023-01-25 16:16:43,904 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Subtask 0 (on host '10.50.89.193') is requesting a file source split 2023-01-25 16:16:43,904 INFO org.apache.flink.connector.file.src.assigners.LocalityAwareSplitAssigner [] - Assigning remote split to requesting host '10': Optional[FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-24-22-d4f2cbbd-79bb-4600-80a5-896241e35856.parquet [0, 109648) hosts=[localhost] ID=0000000002 position=null] 2023-01-25 16:16:43,904 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Assigned split to subtask 0 : FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-24-22-d4f2cbbd-79bb-4600-80a5-896241e35856.parquet [0, 109648) hosts=[localhost] ID=0000000002 position=null 2023-01-25 16:16:43,905 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (a2428909648e0321ca1133dd233cd9b4) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:48,156 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Source Source: sourceTable[1] received split request from parallel task 0 2023-01-25 16:16:48,157 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Subtask 0 (on host '10.50.89.193') is requesting a file source split 2023-01-25 16:16:48,157 INFO org.apache.flink.connector.file.src.assigners.LocalityAwareSplitAssigner [] - Assigning remote split to requesting host '10': Optional[FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-31-52-ea9dcbcf-b534-4914-9745-fe72157e54fb.parquet [0, 32269) hosts=[localhost] ID=0000000003 position=null] 2023-01-25 16:16:48,157 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Assigned split to subtask 0 : FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-31-52-ea9dcbcf-b534-4914-9745-fe72157e54fb.parquet [0, 32269) hosts=[localhost] ID=0000000003 position=null 2023-01-25 16:16:48,415 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Source Source: sourceTable[1] received split request from parallel task 0 2023-01-25 16:16:48,415 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Subtask 0 (on host '10.50.89.193') is requesting a file source split 2023-01-25 16:16:48,415 INFO org.apache.flink.connector.file.src.assigners.LocalityAwareSplitAssigner [] - Assigning remote split to requesting host '10': Optional[FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-37-22-a5e46a71-7ae5-4e26-874b-578ae613732b.parquet [0, 109716) hosts=[localhost] ID=0000000004 position=null] 2023-01-25 16:16:48,415 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Assigned split to subtask 0 : FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-37-22-a5e46a71-7ae5-4e26-874b-578ae613732b.parquet [0, 109716) hosts=[localhost] ID=0000000004 position=null 2023-01-25 16:16:48,658 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Source Source: sourceTable[1] received split request from parallel task 0 2023-01-25 16:16:48,658 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Subtask 0 (on host '10.50.89.193') is requesting a file source split 2023-01-25 16:16:48,658 INFO org.apache.flink.connector.file.src.assigners.LocalityAwareSplitAssigner [] - Assigning remote split to requesting host '10': Optional[FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-12-52-e045b3b8-20e6-4041-8048-898bbfa95624.parquet [0, 109684) hosts=[localhost] ID=0000000001 position=null] 2023-01-25 16:16:48,658 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Assigned split to subtask 0 : FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-12-52-e045b3b8-20e6-4041-8048-898bbfa95624.parquet [0, 109684) hosts=[localhost] ID=0000000001 position=null 2023-01-25 16:16:48,872 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Source Source: sourceTable[1] received split request from parallel task 0 2023-01-25 16:16:48,872 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Subtask 0 (on host '10.50.89.193') is requesting a file source split 2023-01-25 16:16:48,872 INFO org.apache.flink.connector.file.src.assigners.LocalityAwareSplitAssigner [] - Assigning remote split to requesting host '10': Optional[FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-48-52-768746a4-3c50-4f12-ad78-ad91daed1c40.parquet [0, 109712) hosts=[localhost] ID=0000000005 position=null] 2023-01-25 16:16:48,872 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Assigned split to subtask 0 : FileSourceSplit: s3a://bon-log-staging/v3/bon_access_log/account_id=2b9ddaa3-82b3-441f-8826-08686743d421/dt=2023-01-25T15/2023-01-25-15-48-52-768746a4-3c50-4f12-ad78-ad91daed1c40.parquet [0, 109712) hosts=[localhost] ID=0000000005 position=null 2023-01-25 16:16:49,062 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Source Source: sourceTable[1] received split request from parallel task 0 2023-01-25 16:16:49,062 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - Subtask 0 (on host '10.50.89.193') is requesting a file source split 2023-01-25 16:16:49,062 INFO org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator [] - No more splits available for subtask 0 2023-01-25 16:16:49,097 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: sourceTable[1] -> Calc[2] -> HashAggregate[3] (1/1) (a2428909648e0321ca1133dd233cd9b4) switched from RUNNING to FINISHED. 2023-01-25 16:16:49,097 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 39c8a37ddf1096dd2da68216ade26056 2023-01-25 16:16:49,097 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - HashAggregate[5] -> ConstraintEnforcer[6] -> TableToDataSteam -> Map -> Sink: Writer (1/1) (3ff78cca02715461a6157915f7ba08e8) switched from CREATED to SCHEDULED. 2023-01-25 16:16:49,098 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Received resource requirements from job 39c8a37ddf1096dd2da68216ade26056: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}] 2023-01-25 16:16:49,098 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - HashAggregate[5] -> ConstraintEnforcer[6] -> TableToDataSteam -> Map -> Sink: Writer (1/1) (3ff78cca02715461a6157915f7ba08e8) switched from SCHEDULED to DEPLOYING. 2023-01-25 16:16:49,098 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying HashAggregate[5] -> ConstraintEnforcer[6] -> TableToDataSteam -> Map -> Sink: Writer (1/1) (attempt #0) with attempt id 3ff78cca02715461a6157915f7ba08e8 and vertex id c27dcf7b54ef6bfd6cff02ca8870b681_0 to 10.50.89.193:6122-0dd77a @ 10-50-89-193.flink-taskmanager-query-state.optic.svc.cluster.local (dataPort=32835) with allocation id aa1faf216922af1a0d471158553e9bc5 2023-01-25 16:16:49,111 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - HashAggregate[5] -> ConstraintEnforcer[6] -> TableToDataSteam -> Map -> Sink: Writer (1/1) (3ff78cca02715461a6157915f7ba08e8) switched from DEPLOYING to INITIALIZING. 2023-01-25 16:16:49,783 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - HashAggregate[5] -> ConstraintEnforcer[6] -> TableToDataSteam -> Map -> Sink: Writer (1/1) (3ff78cca02715461a6157915f7ba08e8) switched from INITIALIZING to RUNNING. 2023-01-25 16:16:51,812 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - HashAggregate[5] -> ConstraintEnforcer[6] -> TableToDataSteam -> Map -> Sink: Writer (1/1) (3ff78cca02715461a6157915f7ba08e8) switched from RUNNING to CANCELED. 2023-01-25 16:16:51,812 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Clearing resource requirements of job 39c8a37ddf1096dd2da68216ade26056 2023-01-25 16:17:55,030 INFO org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool [] - Releasing slot [aa1faf216922af1a0d471158553e9bc5].