for i in range(2):
p.inversions.iterate(
inverse_problem_configuration="my_inversion",
timeout_in_seconds=360,
ping_interval_in_seconds=10,
)
p.inversions.delete_disposable_files(
inverse_problem_configuration="my_inversion",
data_to_remove=["auxiliary", "gradients", "waveforms"],
)
[2024-03-15 09:25:56,208] INFO: Resuming iteration #1.
[2024-03-15 09:25:56,209] INFO: 1 new tasks have been issued.
[2024-03-15 09:25:56,209] INFO: Processing task `gradient`
[2024-03-15 09:25:56,563] INFO: Submitting job array with 5 jobs ...
Uploading 1 files...
Uploading 1 files...
Uploading 1 files...
Uploading 1 files...
Uploading 1 files...
[2024-03-15 09:25:56,625] INFO: Launched adjoint simulations for 5 events. Please check again to see if they are finished.
[2024-03-15 09:25:56,626] INFO: Some tasks of iteration #1 are still running. Please check again later.
[2024-03-15 09:26:06,628] INFO: Processing task `gradient`
[2024-03-15 09:26:06,837] INFO: 5 events have already been submitted. They will not be submitted again.
[2024-03-15 09:26:07,332] INFO:
Iteration 1: Number of events: 5 chi = 0.006215537864641849 ||g|| = 0.009661529614658408
pred = -0.009679726576114392 ared = -0.011474539275549613 norm_update = 0.953416188995933 tr_radius = 0.9534163991454442
[2024-03-15 09:26:07,342] INFO: 1 new tasks have been issued.
[2024-03-15 09:26:07,342] INFO: Processing task `preconditioner`
[2024-03-15 09:26:07,426] INFO: Some tasks of iteration #1 are still running. Please check again later.
[2024-03-15 09:26:17,446] INFO: Processing task `preconditioner`
[2024-03-15 09:26:17,569] INFO: 1 new tasks have been issued.
[2024-03-15 09:26:17,570] INFO: Processing task `misfit`
[2024-03-15 09:26:17,630] INFO: Submitting job array with 5 jobs ...
[2024-03-15 09:26:17,741] INFO: Launched simulations for 5 events. Please check again to see if they are finished.
[2024-03-15 09:26:17,742] INFO: Some tasks of iteration #1 are still running. Please check again later.
[2024-03-15 09:26:27,775] INFO: Processing task `misfit`
[2024-03-15 09:26:28,585] INFO:
old misfit control group: 0.006215537864641849
new misfit control group: 0.004515143403634327
predicted reduction control group: -0.0008915801779259091
actual reduction control group: -0.0017003944610075221
5 out of 5 event(s) improved the misfit.
[2024-03-15 09:26:28,585] INFO:
Model update accepted.
[2024-03-15 09:26:28,586] INFO: 1 new tasks have been issued.
[2024-03-15 09:26:28,586] INFO: Processing task `finalize_iteration`
[2024-03-15 09:26:28,641] INFO: Succesfully completed iteration #1.
[2024-03-15 09:26:28,643] INFO: Adding new iteration #2.
[2024-03-15 09:26:28,650] INFO: ... searching for obsolete files in project/INVERSIONS/my_inversion/00000
[2024-03-15 09:26:28,653] INFO: ... searching for obsolete files in project/INVERSIONS/my_inversion/00001
[2024-03-15 09:26:28,794] INFO: Freed up 6.0 MB of space.
[2024-03-15 09:26:28,797] INFO: Resuming iteration #2.
[2024-03-15 09:26:28,797] INFO: 1 new tasks have been issued.
[2024-03-15 09:26:28,798] INFO: Processing task `gradient`
[2024-03-15 09:26:29,115] INFO: Submitting job array with 5 jobs ...
Uploading 1 files...
Uploading 1 files...
Uploading 1 files...
Uploading 1 files...
Uploading 1 files...
[2024-03-15 09:26:29,174] INFO: Launched adjoint simulations for 5 events. Please check again to see if they are finished.
[2024-03-15 09:26:29,176] INFO: Some tasks of iteration #2 are still running. Please check again later.
[2024-03-15 09:26:39,178] INFO: Processing task `gradient`
[2024-03-15 09:26:39,388] INFO: 5 events have already been submitted. They will not be submitted again.
[2024-03-15 09:26:39,879] INFO:
Iteration 2: Number of events: 5 chi = 0.004515143403634327 ||g|| = 0.006397878614987469
pred = -0.0008915801779259091 ared = -0.0017003944610075221 norm_update = 0.22181347364063994 tr_radius = 1.9068327982908884
[2024-03-15 09:26:39,895] INFO: 1 new tasks have been issued.
[2024-03-15 09:26:39,895] INFO: Processing task `preconditioner`
[2024-03-15 09:26:39,990] INFO: Some tasks of iteration #2 are still running. Please check again later.
[2024-03-15 09:26:50,021] INFO: Processing task `preconditioner`
[2024-03-15 09:26:50,149] INFO: 1 new tasks have been issued.
[2024-03-15 09:26:50,149] INFO: Processing task `misfit`
[2024-03-15 09:26:50,213] INFO: Submitting job array with 5 jobs ...
[2024-03-15 09:26:50,323] INFO: Launched simulations for 5 events. Please check again to see if they are finished.
[2024-03-15 09:26:50,324] INFO: Some tasks of iteration #2 are still running. Please check again later.
[2024-03-15 09:27:00,364] INFO: Processing task `misfit`
[2024-03-15 09:27:01,029] INFO:
old misfit control group: 0.004515143403634327
new misfit control group: 0.0028780412052226974
predicted reduction control group: -0.0011415243524091146
actual reduction control group: -0.0016371021984116292
5 out of 5 event(s) improved the misfit.
[2024-03-15 09:27:01,029] INFO:
Model update accepted.
[2024-03-15 09:27:01,029] INFO: 1 new tasks have been issued.
[2024-03-15 09:27:01,030] INFO: Processing task `finalize_iteration`
[2024-03-15 09:27:01,104] INFO: Succesfully completed iteration #2.
[2024-03-15 09:27:01,107] INFO: Adding new iteration #3.
[2024-03-15 09:27:01,113] INFO: ... searching for obsolete files in project/INVERSIONS/my_inversion/00000
[2024-03-15 09:27:01,116] INFO: ... searching for obsolete files in project/INVERSIONS/my_inversion/00001
[2024-03-15 09:27:01,201] INFO: ... searching for obsolete files in project/INVERSIONS/my_inversion/00002
[2024-03-15 09:27:01,342] INFO: Freed up 6.0 MB of space.