for i in range(2):
p.inversions.iterate(
inverse_problem_configuration="my_second_inversion",
timeout_in_seconds=360,
ping_interval_in_seconds=10,
)
p.inversions.delete_disposable_files(
inverse_problem_configuration="my_second_inversion",
data_to_remove=["auxiliary", "gradients", "waveforms"],
)
p.viz.nb.inversion(inverse_problem_configuration="my_second_inversion")
[2023-05-29 19:57:01,782] INFO: Resuming iteration #1.
[2023-05-29 19:57:01,783] INFO: 1 new tasks have been issued.
[2023-05-29 19:57:01,783] INFO: Processing task `gradient`
[2023-05-29 19:57:02,246] INFO: Submitting job array with 5 jobs ...
[2023-05-29 19:57:02,370] INFO: Launched adjoint simulations for 5 events. Please check again to see if they are finished.
[2023-05-29 19:57:02,372] INFO: Some tasks of iteration #1 are still running. Please check again later.
[2023-05-29 19:57:12,386] INFO: Processing task `gradient`
[2023-05-29 19:57:12,681] INFO: 5 events have already been submitted. They will not be submitted again.
[2023-05-29 19:57:13,425] INFO:
Iteration 1: Number of events: 5 chi = 0.0066457093447483545 ||g|| = 0.008122033347238987
pred = -0.005735006306668265 ared = -0.011044112201925402 norm_update = 0.7265201743080936 tr_radius = 0.7265201888920712
[2023-05-29 19:57:13,441] INFO: 1 new tasks have been issued.
[2023-05-29 19:57:13,442] INFO: Processing task `preconditioner`
[2023-05-29 19:57:13,596] INFO: Some tasks of iteration #1 are still running. Please check again later.
[2023-05-29 19:57:23,635] INFO: Processing task `preconditioner`
[2023-05-29 19:57:23,859] INFO: 1 new tasks have been issued.
[2023-05-29 19:57:23,860] INFO: Processing task `misfit`
[2023-05-29 19:57:23,955] INFO: Submitting job array with 5 jobs ...
[2023-05-29 19:57:24,139] INFO: Launched simulations for 5 events. Please check again to see if they are finished.
[2023-05-29 19:57:24,141] INFO: Some tasks of iteration #1 are still running. Please check again later.
[2023-05-29 19:57:34,201] INFO: Processing task `misfit`
[2023-05-29 19:57:35,174] INFO:
old misfit control group: 0.0066457093447483545
new misfit control group: 0.0028967694956960307
predicted reduction control group: -0.0027947378751068505
actual reduction control group: -0.003748939849052324
5 out of 5 event(s) improved the misfit.
[2023-05-29 19:57:35,175] INFO:
Model update accepted.
[2023-05-29 19:57:35,176] INFO: 1 new tasks have been issued.
[2023-05-29 19:57:35,176] INFO: Processing task `finalize_iteration`
[2023-05-29 19:57:35,271] INFO: Succesfully completed iteration #1.
[2023-05-29 19:57:35,275] INFO: Adding new iteration #2.
[2023-05-29 19:57:35,285] INFO: ... searching for obsolete files in project/INVERSIONS/my_second_inversion/00000
[2023-05-29 19:57:35,288] INFO: ... searching for obsolete files in project/INVERSIONS/my_second_inversion/00001
[2023-05-29 19:57:35,401] INFO: Freed up 4.4 MB of space.
[2023-05-29 19:57:35,407] INFO: Resuming iteration #2.
[2023-05-29 19:57:35,407] INFO: 1 new tasks have been issued.
[2023-05-29 19:57:35,408] INFO: Processing task `gradient`
[2023-05-29 19:57:35,852] INFO: Submitting job array with 5 jobs ...
[2023-05-29 19:57:35,963] INFO: Launched adjoint simulations for 5 events. Please check again to see if they are finished.
[2023-05-29 19:57:35,964] INFO: Some tasks of iteration #2 are still running. Please check again later.
[2023-05-29 19:57:45,978] INFO: Processing task `gradient`
[2023-05-29 19:57:46,270] INFO: 5 events have already been submitted. They will not be submitted again.
[2023-05-29 19:57:47,184] INFO:
Iteration 2: Number of events: 5 chi = 0.0028967694956960307 ||g|| = 0.0033621126637955632
pred = -0.0027947378751068505 ared = -0.003748939849052324 norm_update = 0.7176637300170433 tr_radius = 1.4530403777841423
[2023-05-29 19:57:47,214] INFO: 1 new tasks have been issued.
[2023-05-29 19:57:47,215] INFO: Processing task `preconditioner`
[2023-05-29 19:57:47,383] INFO: Some tasks of iteration #2 are still running. Please check again later.
[2023-05-29 19:57:57,441] INFO: Processing task `preconditioner`
[2023-05-29 19:57:57,669] INFO: 1 new tasks have been issued.
[2023-05-29 19:57:57,670] INFO: Processing task `misfit`
[2023-05-29 19:57:57,765] INFO: Submitting job array with 5 jobs ...
[2023-05-29 19:57:57,953] INFO: Launched simulations for 5 events. Please check again to see if they are finished.
[2023-05-29 19:57:57,955] INFO: Some tasks of iteration #2 are still running. Please check again later.
[2023-05-29 19:58:08,038] INFO: Processing task `misfit`
[2023-05-29 19:58:09,011] INFO:
old misfit control group: 0.0028967694956960307
new misfit control group: 0.002407850346859309
predicted reduction control group: -0.0002203629212521284
actual reduction control group: -0.0004889191488367216
5 out of 5 event(s) improved the misfit.
[2023-05-29 19:58:09,012] INFO:
Model update accepted.
[2023-05-29 19:58:09,013] INFO: 1 new tasks have been issued.
[2023-05-29 19:58:09,014] INFO: Processing task `finalize_iteration`
[2023-05-29 19:58:09,145] INFO: Succesfully completed iteration #2.
[2023-05-29 19:58:09,150] INFO: Adding new iteration #3.
[2023-05-29 19:58:09,161] INFO: ... searching for obsolete files in project/INVERSIONS/my_second_inversion/00000
[2023-05-29 19:58:09,165] INFO: ... searching for obsolete files in project/INVERSIONS/my_second_inversion/00001
[2023-05-29 19:58:09,281] INFO: ... searching for obsolete files in project/INVERSIONS/my_second_inversion/00002
[2023-05-29 19:58:09,393] INFO: Freed up 4.4 MB of space.
