[2023-03-17 20:00:14,494] INFO: Adding new iteration #0.
[2023-03-17 20:00:14,505] INFO: Resuming iteration #0.
[2023-03-17 20:00:14,506] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:14,507] INFO: Processing task `misfit_and_gradient`
[2023-03-17 20:00:14,529] INFO: The following events have been simulated before, but checkpoints are not available for this combination of `site_name` and `ranks_per_job` and wavefield compression settings. They will be run again: ['event']
[2023-03-17 20:00:14,544] INFO: Submitting job ...
[2023-03-17 20:00:14,608] INFO: Launched simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:14,609] INFO: Some tasks of iteration #0 are still running. Please check again later.
[2023-03-17 20:00:15,624] INFO: Processing task `misfit_and_gradient`
[2023-03-17 20:00:16,013] INFO: Submitting job ...
[2023-03-17 20:00:16,089] INFO: Launched adjoint simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:16,090] INFO: Some tasks of iteration #0 are still running. Please check again later.
[2023-03-17 20:00:17,094] INFO: Processing task `misfit_and_gradient`
[2023-03-17 20:00:17,193] INFO: 1 events have already been submitted. They will not be submitted again.
[2023-03-17 20:00:17,356] INFO:
Iteration 0: Number of events: 1 chi = 3.715653793726883e-10 ||g|| = 3.3910361185905376e-12
pred = --- ared = --- norm_update = --- tr_radius = ---
[2023-03-17 20:00:17,362] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:17,363] INFO: Processing task `misfit`
[2023-03-17 20:00:17,400] INFO: Submitting job ...
[2023-03-17 20:00:17,464] INFO: Launched simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:17,465] INFO: Some tasks of iteration #0 are still running. Please check again later.
[2023-03-17 20:00:18,489] INFO: Processing task `misfit`
[2023-03-17 20:00:18,566] INFO: Some tasks of iteration #0 are still running. Please check again later.
[2023-03-17 20:00:19,579] INFO: Processing task `misfit`
[2023-03-17 20:00:19,708] INFO:
old misfit control group: 3.715653793726883e-10
new misfit control group: 1.0738902017002148e-10
predicted reduction control group: -2.1653942690707703e-10
actual reduction control group: -2.641763592026668e-10
1 out of 1 event(s) improved the misfit.
[2023-03-17 20:00:19,709] INFO:
Model update accepted.
[2023-03-17 20:00:19,710] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:19,710] INFO: Processing task `finalize_iteration`
[2023-03-17 20:00:19,731] INFO: Succesfully completed iteration #0.
[2023-03-17 20:00:19,734] INFO: Adding new iteration #1.
[2023-03-17 20:00:19,740] INFO: Resuming iteration #1.
[2023-03-17 20:00:19,741] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:19,742] INFO: Processing task `gradient`
[2023-03-17 20:00:19,824] INFO: Submitting job ...
[2023-03-17 20:00:19,888] INFO: Launched adjoint simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:19,889] INFO: Some tasks of iteration #1 are still running. Please check again later.
[2023-03-17 20:00:20,894] INFO: Processing task `gradient`
[2023-03-17 20:00:20,952] INFO: 1 events have already been submitted. They will not be submitted again.
[2023-03-17 20:00:21,146] INFO:
Iteration 1: Number of events: 1 chi = 1.0738902017002148e-10 ||g|| = 1.0372298896016516e-12
pred = -2.1653942690707703e-10 ared = -2.641763592026668e-10 norm_update = 127.71284016702265 tr_radius = 127.71284016702259
[2023-03-17 20:00:21,161] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:21,162] INFO: Processing task `misfit`
[2023-03-17 20:00:21,199] INFO: Submitting job ...
[2023-03-17 20:00:21,269] INFO: Launched simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:21,270] INFO: Some tasks of iteration #1 are still running. Please check again later.
[2023-03-17 20:00:22,296] INFO: Processing task `misfit`
[2023-03-17 20:00:22,459] INFO:
old misfit control group: 1.0738902017002148e-10
new misfit control group: 7.296709328106128e-11
predicted reduction control group: -1.8864361819323874e-11
actual reduction control group: -3.4421926888960195e-11
1 out of 1 event(s) improved the misfit.
[2023-03-17 20:00:22,460] INFO:
Model update accepted.
[2023-03-17 20:00:22,460] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:22,461] INFO: Processing task `finalize_iteration`
[2023-03-17 20:00:22,482] INFO: Succesfully completed iteration #1.
[2023-03-17 20:00:22,484] INFO: Adding new iteration #2.
[2023-03-17 20:00:22,492] INFO: Resuming iteration #2.
[2023-03-17 20:00:22,493] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:22,493] INFO: Processing task `gradient`
[2023-03-17 20:00:22,582] INFO: Submitting job ...
[2023-03-17 20:00:22,648] INFO: Launched adjoint simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:22,650] INFO: Some tasks of iteration #2 are still running. Please check again later.
[2023-03-17 20:00:23,655] INFO: Processing task `gradient`
[2023-03-17 20:00:23,727] INFO: 1 events have already been submitted. They will not be submitted again.
[2023-03-17 20:00:23,812] INFO: Some simulations are still running. Please check again to see if they are finished.
[2023-03-17 20:00:23,813] INFO: Some tasks of iteration #2 are still running. Please check again later.
[2023-03-17 20:00:24,818] INFO: Processing task `gradient`
[2023-03-17 20:00:24,882] INFO: 1 events have already been submitted. They will not be submitted again.
[2023-03-17 20:00:25,037] INFO:
Iteration 2: Number of events: 1 chi = 7.296709328106128e-11 ||g|| = 8.841253896552173e-13
pred = -1.8864361819323874e-11 ared = -3.4421926888960195e-11 norm_update = 38.0551384532734 tr_radius = 255.42568033404518
[2023-03-17 20:00:25,063] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:25,064] INFO: Processing task `misfit`
[2023-03-17 20:00:25,099] INFO: Submitting job ...
[2023-03-17 20:00:25,164] INFO: Launched simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:25,166] INFO: Some tasks of iteration #2 are still running. Please check again later.
[2023-03-17 20:00:26,189] INFO: Processing task `misfit`
[2023-03-17 20:00:26,327] INFO:
old misfit control group: 7.296709328106128e-11
new misfit control group: 7.299258012354608e-12
predicted reduction control group: -5.3947822233791415e-11
actual reduction control group: -6.566783526870668e-11
1 out of 1 event(s) improved the misfit.
[2023-03-17 20:00:26,328] INFO:
Model update accepted.
[2023-03-17 20:00:26,329] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:26,329] INFO: Processing task `finalize_iteration`
[2023-03-17 20:00:26,352] INFO: Succesfully completed iteration #2.
[2023-03-17 20:00:26,355] INFO: Adding new iteration #3.
[2023-03-17 20:00:26,368] INFO: Resuming iteration #3.
[2023-03-17 20:00:26,369] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:26,370] INFO: Processing task `gradient`
[2023-03-17 20:00:26,451] INFO: Submitting job ...
[2023-03-17 20:00:26,517] INFO: Launched adjoint simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:26,518] INFO: Some tasks of iteration #3 are still running. Please check again later.
[2023-03-17 20:00:27,528] INFO: Processing task `gradient`
[2023-03-17 20:00:27,588] INFO: 1 events have already been submitted. They will not be submitted again.
[2023-03-17 20:00:27,680] INFO: Some simulations are still running. Please check again to see if they are finished.
[2023-03-17 20:00:27,681] INFO: Some tasks of iteration #3 are still running. Please check again later.
[2023-03-17 20:00:28,686] INFO: Processing task `gradient`
[2023-03-17 20:00:28,753] INFO: 1 events have already been submitted. They will not be submitted again.
[2023-03-17 20:00:28,946] INFO:
Iteration 3: Number of events: 1 chi = 7.299258012354608e-12 ||g|| = 2.1081352436605497e-13
pred = -5.3947822233791415e-11 ared = -6.566783526870668e-11 norm_update = 127.7128401670221 tr_radius = 127.71284016702259
[2023-03-17 20:00:28,976] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:28,976] INFO: Processing task `misfit`
[2023-03-17 20:00:29,010] INFO: Submitting job ...
[2023-03-17 20:00:29,074] INFO: Launched simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:29,075] INFO: Some tasks of iteration #3 are still running. Please check again later.
[2023-03-17 20:00:30,098] INFO: Processing task `misfit`
[2023-03-17 20:00:30,252] INFO:
old misfit control group: 7.299258012354608e-12
new misfit control group: 3.854024430459155e-12
predicted reduction control group: -3.625596913405018e-12
actual reduction control group: -3.4452335818954526e-12
1 out of 1 event(s) improved the misfit.
[2023-03-17 20:00:30,253] INFO:
Model update accepted.
[2023-03-17 20:00:30,253] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:30,254] INFO: Processing task `finalize_iteration`
[2023-03-17 20:00:30,277] INFO: Succesfully completed iteration #3.
[2023-03-17 20:00:30,280] INFO: Adding new iteration #4.
[2023-03-17 20:00:30,293] INFO: Resuming iteration #4.
[2023-03-17 20:00:30,294] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:30,294] INFO: Processing task `gradient`
[2023-03-17 20:00:30,382] INFO: Submitting job ...
[2023-03-17 20:00:30,444] INFO: Launched adjoint simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:30,446] INFO: Some tasks of iteration #4 are still running. Please check again later.
[2023-03-17 20:00:31,451] INFO: Processing task `gradient`
[2023-03-17 20:00:31,514] INFO: 1 events have already been submitted. They will not be submitted again.
[2023-03-17 20:00:31,599] INFO: Some simulations are still running. Please check again to see if they are finished.
[2023-03-17 20:00:31,600] INFO: Some tasks of iteration #4 are still running. Please check again later.
[2023-03-17 20:00:32,605] INFO: Processing task `gradient`
[2023-03-17 20:00:32,664] INFO: 1 events have already been submitted. They will not be submitted again.
[2023-03-17 20:00:32,856] INFO:
Iteration 4: Number of events: 1 chi = 3.854024430459155e-12 ||g|| = 5.4926565444312184e-14
pred = -3.625596913405018e-12 ared = -3.4452335818954526e-12 norm_update = 37.63483481452526 tr_radius = 255.42568033404518
[2023-03-17 20:00:32,893] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:32,894] INFO: Processing task `misfit`
[2023-03-17 20:00:32,928] INFO: Submitting job ...
[2023-03-17 20:00:32,993] INFO: Launched simulations for 1 events. Please check again to see if they are finished.
[2023-03-17 20:00:32,994] INFO: Some tasks of iteration #4 are still running. Please check again later.
[2023-03-17 20:00:34,016] INFO: Processing task `misfit`
[2023-03-17 20:00:34,157] INFO:
old misfit control group: 3.854024430459155e-12
new misfit control group: 3.660842563123943e-12
predicted reduction control group: -1.1396035952939152e-13
actual reduction control group: -1.9318186733521243e-13
1 out of 1 event(s) improved the misfit.
[2023-03-17 20:00:34,158] INFO:
Model update accepted.
[2023-03-17 20:00:34,158] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:34,159] INFO: Processing task `finalize_iteration`
[2023-03-17 20:00:34,198] INFO: Succesfully completed iteration #4.
[2023-03-17 20:00:34,203] INFO: Adding new iteration #5.
[2023-03-17 20:00:34,220] INFO: Resuming iteration #5.
[2023-03-17 20:00:34,220] INFO: 1 new tasks have been issued.
[2023-03-17 20:00:34,221] INFO: Processing task `g