[1]:
%matplotlib inline
CC vs PCC — Side-by-side Comparison
This notebook builds a minimal MSNoise project from scratch and runs two parallel correlation lineages on the same dataset:
Step name |
Method |
Notes |
|---|---|---|
|
— |
shared preprocessing |
|
CC |
standard geometrically-normalised |
|
PCC |
phase cross-correlation (v=2) |
|
— |
shared band-pass filter |
|
linear |
shared stacking config |
Both cc_1 and cc_2 feed the same filter_1 → stack_1 steps. The final section retrieves stacked CCFs from both lineages and plots them side-by-side for a direct comparison.
Prerequisites
MSNoise installed (with the PCC patch applied).
A one-day MiniSEED dataset for at least two stations. The classic MSNoise tutorial dataset (network
YA, stationsUV05,UV06,UV10, day 2010-09-01 / Julian day 244, stored in PDF layout) is used as the reference, but any SDS or PDF archive works — just adjust the USER SETTINGS cell below.
0 · Imports
[2]:
%matplotlib inline
import os
import shutil
import logging
import tempfile
import numpy as np
import matplotlib.pyplot as plt
# MSNoise core
from msnoise.core.db import connect, create_database_inifile
from msnoise.core.config import (
create_config_set, update_config
)
from msnoise.core.stations import update_station
from msnoise.core.workflow import (
create_workflow_steps_from_config_sets,
create_workflow_links_from_steps,
get_workflow_steps,
reset_jobs
)
from msnoise.msnoise_table_def import declare_tables, DataAvailability
from msnoise.results import MSNoiseResult
# MSNoise compute steps
from msnoise.s01_scan_archive import main as scan_archive
from msnoise.s02_new_jobs import main as new_jobs
from msnoise.s02_preprocessing import main as preprocess
from msnoise.s03_compute_no_rotation import main as compute_cc
from msnoise.s04_stack_mov import main as stack_mov
logging.basicConfig(level=logging.WARNING) # suppress INFO noise in notebook
1 · User Settings
Edit this cell to match your dataset.
The defaults match the classic MSNoise tutorial dataset (YA network, stations UV05/UV06/UV10, PDF archive layout).
[3]:
# ── Path to your waveform archive ────────────────────────────────────────────
# Can be an absolute path or relative.
# The installer will store it verbatim in the DataSource table.
DATA_PATH = r"C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data"
# ── Archive layout ────────────────────────────────────────────────────────────
# "PDF" → YEAR/STA/CHAN.D/NET.STA.LOC.CHAN.D.YEAR.JDAY
# "SDS" → YEAR/NET/STA/CHAN.D/NET.STA.LOC.CHAN.D.YEAR.JDAY
DATA_STRUCTURE = "PDF" # ← "SDS" or "PDF"
# ── Network / channel filter used when scanning the archive ──────────────────
NETWORK_CODE = "YA"
CHANNELS = "HHZ"
# ── Date range to process (ISO format, inclusive) ────────────────────────────
STARTDATE = "2010-09-01"
ENDDATE = "2010-09-01"
# ── Station coordinates (net, sta, lon°E, lat°N, elev_m) ─────────────────────
# Fill in your actual stations. These are the YA tutorial stations.
STATIONS = [
("YA", "UV05", 29.735, -17.817, 1174.0),
("YA", "UV06", 29.785, -17.827, 1162.0),
("YA", "UV10", 29.790, -17.847, 1180.0),
]
# ── CC config (shared by both cc_1 and cc_2) ─────────────────────────────────
CC_SAMPLING_RATE = 20.0 # Hz — resample target
MAXLAG = 120.0 # seconds
CORR_DURATION = 1800.0 # seconds per window
COMPONENTS = "ZZ"
# ── Filter passband ───────────────────────────────────────────────────────────
FREQMIN = 0.1 # Hz
FREQMAX = 4.0 # Hz
# ── Stack config ──────────────────────────────────────────────────────────────
MOV_STACK = "(('1h','1h'),)" # 1-hour stack
# ── Working directory (project folder) ───────────────────────────────────────
# A fresh temporary directory is created automatically.
# Set to a fixed path if you want a persistent project between kernel restarts.
WORK_DIR = None # None → auto temp dir
2 · Create Project Directory and MSNoise Database
[4]:
if WORK_DIR is None:
WORK_DIR = tempfile.mkdtemp(prefix="msnoise_ccvspcc_")
print(f"Created temporary project directory: {WORK_DIR}")
else:
os.makedirs(WORK_DIR, exist_ok=True)
print(f"Using project directory: {WORK_DIR}")
os.chdir(WORK_DIR)
# Initialise SQLite database
create_database_inifile(
tech=1,
hostname=os.path.join(WORK_DIR, "msnoise.sqlite"),
database="", username="", password="", prefix="",
)
# Create all tables
db = connect()
declare_tables().Base.metadata.create_all(db.get_bind())
print("Database created:", os.path.join(WORK_DIR, "msnoise.sqlite"))
Created temporary project directory: C:\Users\tlecocq\AppData\Local\Temp\msnoise_ccvspcc_qxiyrqnw
Database created: C:\Users\tlecocq\AppData\Local\Temp\msnoise_ccvspcc_qxiyrqnw\msnoise.sqlite
3 · Create Default Config Sets and Workflow Skeleton
We create one config set for every category except cc, for which we will create two (cc_1 = CC, cc_2 = PCC).
[5]:
ALL_CATEGORIES = [
"global", "preprocess", "filter", "stack", "refstack",
"mwcs", "mwcs_dtt", "mwcs_dtt_dvv",
"stretching", "stretching_dvv",
"wavelet", "wavelet_dtt", "wavelet_dtt_dvv",
"psd", "psd_rms",
]
for cat in ALL_CATEGORIES:
sn = create_config_set(db, cat)
print(f" {cat}_1 created (set_number={sn})")
# CC set 1 — standard cross-correlation
cc1_sn = create_config_set(db, "cc")
print(f" cc_{cc1_sn} created ← standard CC")
# CC set 2 — phase cross-correlation
cc2_sn = create_config_set(db, "cc")
print(f" cc_{cc2_sn} created ← PCC")
db.commit()
global_1 created (set_number=1)
preprocess_1 created (set_number=1)
filter_1 created (set_number=1)
stack_1 created (set_number=1)
refstack_1 created (set_number=1)
mwcs_1 created (set_number=1)
mwcs_dtt_1 created (set_number=1)
mwcs_dtt_dvv_1 created (set_number=1)
stretching_1 created (set_number=1)
stretching_dvv_1 created (set_number=1)
wavelet_1 created (set_number=1)
wavelet_dtt_1 created (set_number=1)
wavelet_dtt_dvv_1 created (set_number=1)
psd_1 created (set_number=1)
psd_rms_1 created (set_number=1)
cc_1 created ← standard CC
cc_2 created ← PCC
4 · Configure Global Parameters
[6]:
OUTPUT_FOLDER = os.path.join(WORK_DIR, "OUTPUT")
global_params = {
"output_folder": OUTPUT_FOLDER,
"startdate": STARTDATE,
"enddate": ENDDATE,
"hpc": "N", # propagate_downstream fires automatically
}
for k, v in global_params.items():
update_config(db, k, v, category="global", set_number=1)
print(f" global.{k} = {v!r}")
global.output_folder = 'C:\\Users\\tlecocq\\AppData\\Local\\Temp\\msnoise_ccvspcc_qxiyrqnw\\OUTPUT'
global.startdate = '2010-09-01'
global.enddate = '2010-09-01'
global.hpc = 'N'
5 · Configure the Two CC Sets
[7]:
# ── Shared CC parameters ──────────────────────────────────────────────────────
cc_shared = {
"cc_sampling_rate": str(CC_SAMPLING_RATE),
"maxlag": str(MAXLAG),
"corr_duration": str(CORR_DURATION),
"components_to_compute": COMPONENTS,
"keep_all": "Y",
"keep_days": "Y",
"whitening": "A", # whiten all inter-station pairs
"winsorizing": "3.0", # 3× RMS clipping
"clip_after_whiten": "N",
"stack_method": "linear",
}
for k, v in cc_shared.items():
update_config(db, k, v, category="cc", set_number=cc1_sn)
update_config(db, k, v, category="cc", set_number=cc2_sn)
# ── cc_1: standard CC ─────────────────────────────────────────────────────────
update_config(db, "cc_type", "CC", category="cc", set_number=cc1_sn)
update_config(db, "whitening", "A", category="cc", set_number=cc1_sn)
print(f"cc_{cc1_sn}: cc_type = CC (standard cross-correlation)")
# ── cc_2: PCC (Phase Cross-Correlation v=2) ──────────────────────────────────
#
# What happens in the PCC branch of s03:
# 1. The filter_N bandpass is applied to the time-domain traces so each
# filter produces a genuinely distinct CCF (CC does this via whiten2's
# frequency window; PCC needs it done explicitly).
# 2. If whitening != "N", whiten2 is also applied within the band BEFORE
# AmpNorm. This flattens the spectrum so the phase signal is broadband
# across the full filter passband rather than dominated by the most
# energetic frequency in the band ("spectrally whitened PCC",
# Ventosa 2019 §3.1). With whitening="N" you get the classic PCC
# result whose bandwidth is set solely by the preprocessing bandpass.
#
# Winsorising: AmpNorm already discards all amplitude information per-sample,
# so winsorising (clipping at N×RMS) is redundant for PCC. Disable it.
update_config(db, "cc_type", "PCC", category="cc", set_number=cc2_sn)
update_config(db, "winsorizing", "0", category="cc", set_number=cc2_sn)
update_config(db, "whitening", "A", category="cc", set_number=cc2_sn)
print(f"cc_{cc2_sn}: cc_type = PCC (phase cross-correlation v=2)")
print(f" winsorizing=0 (redundant: AmpNorm already discards amplitude)")
print(f" whitening=A (spectral whitening within band before AmpNorm)")
db.commit()
cc_1: cc_type = CC (standard cross-correlation)
cc_2: cc_type = PCC (phase cross-correlation v=2)
winsorizing=0 (redundant: AmpNorm already discards amplitude)
whitening=A (spectral whitening within band before AmpNorm)
6 · Configure Filter and Stack
[8]:
update_config(db, "freqmin", str(FREQMIN), category="filter", set_number=1)
update_config(db, "freqmax", str(FREQMAX), category="filter", set_number=1)
update_config(db, "CC", "Y", category="filter", set_number=1)
update_config(db, "mov_stack", MOV_STACK, category="stack", set_number=1)
update_config(db, "ref_begin", STARTDATE, category="refstack", set_number=1)
update_config(db, "ref_end", ENDDATE, category="refstack", set_number=1)
db.commit()
print(f"filter_1: {FREQMIN}–{FREQMAX} Hz")
print(f"stack_1: {MOV_STACK}")
filter_1: 0.1–4.0 Hz
stack_1: (('1h','1h'),)
7 · Configure DataSource and Station Table
[9]:
# Create the default DataSource (the installer normally does this; when
# initialising the DB manually we must insert it ourselves).
DataSource = declare_tables().DataSource
ds = DataSource(
name="local",
uri=os.path.realpath(DATA_PATH),
data_structure=DATA_STRUCTURE,
auth_env="MSNOISE",
network_code=NETWORK_CODE,
channels=CHANNELS,
)
db.add(ds)
db.commit()
print(f"DataSource created: uri={os.path.realpath(DATA_PATH)!r}")
print(f" data_structure={DATA_STRUCTURE!r}")
# Add stations
for net, sta, lon, lat, elev in STATIONS:
update_station(db, net=net, sta=sta, X=lon, Y=lat, altitude=elev,
coordinates="DEG", used=1)
print(f" Added station {net}.{sta} ({lon:.3f}°E, {lat:.3f}°N)")
db.commit()
DataSource created: uri='C:\\Users\\tlecocq\\AppData\\Local\\msnoise-testdata\\msnoise-testdata\\Cache\\1.1\\classic\\data'
data_structure='PDF'
Added station YA.UV05 (29.735°E, -17.817°N)
Added station YA.UV06 (29.785°E, -17.827°N)
Added station YA.UV10 (29.790°E, -17.847°N)
8 · Build the Workflow Graph
The topology we want:
preprocess_1 ──► cc_1 ──► filter_1 ──► stack_1
└──► cc_2 ──► ↑
Both cc steps feed the same filter_1 and stack_1.
[10]:
# Create WorkflowSteps from all config sets (auto-discovers preprocess_1,
# cc_1, cc_2, filter_1, stack_1, …)
created, existing, err = create_workflow_steps_from_config_sets(db)
assert err is None, f"Error creating workflow steps: {err}"
print(f"Workflow steps: {created} created, {existing} already existed")
# Auto-link following MSNoise's canonical dependency rules
created_links, existing_links, err = create_workflow_links_from_steps(db)
assert err is None, f"Error creating workflow links: {err}"
print(f"Workflow links: {created_links} created, {existing_links} already existed")
# Verify the topology
steps = {s.step_name: s for s in get_workflow_steps(db)}
print("\nWorkflow steps present:", sorted(steps.keys()))
Workflow steps: 17 created, 0 already existed
Workflow links: 18 created, 0 already existed
Workflow steps present: ['cc_1', 'cc_2', 'filter_1', 'global_1', 'mwcs_1', 'mwcs_dtt_1', 'mwcs_dtt_dvv_1', 'preprocess_1', 'psd_1', 'psd_rms_1', 'refstack_1', 'stack_1', 'stretching_1', 'stretching_dvv_1', 'wavelet_1', 'wavelet_dtt_1', 'wavelet_dtt_dvv_1']
9 · Scan Archive and Seed Jobs
[11]:
# Scan the waveform archive → populate DataAvailability table
scan_archive(init=True, threads=1)
# Update loc/chan on Station rows from DataAvailability
from sqlalchemy import text as _text
_db2 = connect()
for sta in _db2.query(declare_tables().Station):
data = _db2.query(DataAvailability). \
filter(_text("net=:net")).filter(_text("sta=:sta")). \
group_by(DataAvailability.net, DataAvailability.sta,
DataAvailability.loc, DataAvailability.chan). \
params(net=sta.net, sta=sta.sta).all()
sta.used_location_codes = ",".join(sorted({d.loc for d in data}))
sta.used_channel_names = ",".join(sorted({d.chan for d in data}))
_db2.commit()
_db2.close()
# Verify availability
_db3 = connect()
n_da = _db3.query(DataAvailability).count()
print(f"DataAvailability rows: {n_da}")
_db3.close()
# Seed initial jobs (creates preprocess_1 T-jobs)
new_jobs(init=True)
2026-04-29 22:12:53.143451 msnoise [INFO]: *** Starting: Scan Archive ***
2026-04-29 22:12:53.144938 msnoise [INFO]: Will work on 1 thread(s)
2026-04-29 22:12:53.144938 msnoise [INFO]: Initializing: updating availability using the whole archive (should be run only once)
2026-04-29 22:12:53.149053 msnoise [INFO]: Will search for files between 2010-09-01 and 2010-09-01.
2026-04-29 22:12:53.155987 msnoise [INFO]: DataSource 'local' (C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data): 3 directories to scan
2026-04-29 22:12:53.156989 msnoise [INFO]: Scanning DataSource 'local' ...
2026-04-29 22:12:53.159119 msnoise [INFO]: Scanning 3 directories...
2026-04-29 22:12:53.162119 msnoise [pid 15020][DEBUG]: scanning dir C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV05/HHZ.D
2026-04-29 22:12:53.163116 msnoise [pid 15020][DEBUG]: Found 1 files in C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV05/HHZ.D
2026-04-29 22:12:53.343320 msnoise [pid 15020][DEBUG]: read file C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV05/HHZ.D\YA.UV05.00.HHZ.D.2010.244, id: YA.UV05.00.HHZ
2026-04-29 22:12:53.351306 msnoise [pid 15020][INFO]: C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV05/HHZ.D: Added 1 | Modified 0 | Unchanged 0
2026-04-29 22:12:53.352458 msnoise [pid 15020][DEBUG]: scanning dir C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV06/HHZ.D
2026-04-29 22:12:53.353592 msnoise [pid 15020][DEBUG]: Found 1 files in C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV06/HHZ.D
2026-04-29 22:12:53.378886 msnoise [pid 15020][DEBUG]: read file C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV06/HHZ.D\YA.UV06.00.HHZ.D.2010.244, id: YA.UV06.00.HHZ
2026-04-29 22:12:53.386770 msnoise [pid 15020][INFO]: C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV06/HHZ.D: Added 1 | Modified 0 | Unchanged 0
2026-04-29 22:12:53.387800 msnoise [pid 15020][DEBUG]: scanning dir C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV10/HHZ.D
2026-04-29 22:12:53.387800 msnoise [pid 15020][DEBUG]: Found 1 files in C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV10/HHZ.D
2026-04-29 22:12:53.416759 msnoise [pid 15020][DEBUG]: read file C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV10/HHZ.D\YA.UV10.00.HHZ.D.2010.244, id: YA.UV10.00.HHZ
2026-04-29 22:12:53.424803 msnoise [pid 15020][INFO]: C:\Users\tlecocq\AppData\Local\msnoise-testdata\msnoise-testdata\Cache\1.1\classic\data\2010/UV10/HHZ.D: Added 1 | Modified 0 | Unchanged 0
2026-04-29 22:12:53.425878 msnoise [pid 15020][INFO]: *** Finished: Scan Archive ***
2026-04-29 22:12:53.425878 msnoise [pid 15020][INFO]: It took 0.28 seconds
DataAvailability rows: 3
2026-04-29 22:12:53.481354 msnoise [pid 15020][INFO]: *** Starting: New Jobs (Workflow-aware) ***
2026-04-29 22:12:53.482889 msnoise [pid 15020][DEBUG]: Checking plugins' entry points
2026-04-29 22:12:53.488197 msnoise [pid 15020][INFO]: Found 2 preprocess workflow steps
2026-04-29 22:12:53.489231 msnoise [pid 15020][DEBUG]: - preprocess_1 (ID: 2)
2026-04-29 22:12:53.489231 msnoise [pid 15020][DEBUG]: - psd_1 (ID: 5)
2026-04-29 22:12:53.489231 msnoise [pid 15020][INFO]: Scanning New/Modified files
2026-04-29 22:12:53.504796 msnoise [pid 15020][DEBUG]: Inserting/Updating 6 jobs
2026-04-29 22:12:53.529812 msnoise [pid 15020][INFO]: Inserted 12 jobs
2026-04-29 22:12:53.529812 msnoise [pid 15020][INFO]: *** Finished: New Jobs (Workflow-aware) ***
2026-04-29 22:12:53.541208 msnoise [pid 15020][DEBUG]: hpc=False: preprocess→cc propagation handled by propagate_downstream()
[11]:
12
10 · Run the Pipeline
Steps run sequentially in-process. For large datasets use msnoise CLI or HPC submission instead.
[12]:
print("── preprocess ─────────────────────────────────────────────────────")
preprocess()
print("── new_jobs --after preprocess ────────────────────────────────────")
new_jobs(after="preprocess")
print("── compute_cc (cc_1 = CC and cc_2 = PCC) ───────────────────────")
compute_cc()
print("── new_jobs --after cc ────────────────────────────────────────────")
new_jobs(after="cc")
print("── stack_mov ──────────────────────────────────────────────────────")
stack_mov(stype="mov")
print("── new_jobs --after stack ─────────────────────────────────────────")
new_jobs(after="stack")
print("Done.")
── preprocess ─────────────────────────────────────────────────────
2026-04-29 22:12:53.551377 msnoise [pid 15020][INFO]: *** Starting: Preprocessing Step ***
2026-04-29 22:12:53.596306 msnoise [INFO]: New PREPROCESS batch: pair=YA.UV05.00 n=3 group_by=day_lineage lineage=preprocess_1
2026-04-29 22:12:53.596306 msnoise [INFO]: Processing 3 jobs for step 'preprocess_1' on 2010-09-01
2026-04-29 22:12:53.609257 msnoise [pid 15020][DEBUG]: Processing stations (local, DataSource 'local'): ['YA.UV05.00', 'YA.UV06.00', 'YA.UV10.00']
2026-04-29 22:12:53.609257 msnoise [pid 15020][DEBUG]: *** Starting: preprocessing ***
2026-04-29 22:12:53.619433 msnoise [pid 15020][DEBUG]: YA.UV05.00.Z Reading 1 Files
2026-04-29 22:12:54.943748 msnoise [pid 15020][DEBUG]: YA.UV06.00.Z Reading 1 Files
2026-04-29 22:12:55.860131 msnoise [pid 15020][DEBUG]: YA.UV10.00.Z Reading 1 Files
2026-04-29 22:12:56.753364 msnoise [pid 15020][INFO]: Finished preprocessing
2026-04-29 22:12:56.910580 msnoise [pid 15020][INFO]: Saved 3 preprocessed file(s)
2026-04-29 22:12:56.916373 msnoise [pid 15020][INFO]: Creating CC jobs from completed preprocess jobs
2026-04-29 22:12:56.930460 msnoise [pid 15020][DEBUG]: Processing preprocess step: preprocess_1
2026-04-29 22:12:56.934464 msnoise [pid 15020][DEBUG]: Creating jobs for CC step: cc_1
2026-04-29 22:12:56.955814 msnoise [pid 15020][DEBUG]: Creating jobs for CC step: cc_2
2026-04-29 22:12:57.003046 msnoise [pid 15020][INFO]: *** Finished: Preprocessing Step ***
── new_jobs --after preprocess ────────────────────────────────────
2026-04-29 22:12:57.005075 msnoise [pid 15020][INFO]: *** Starting: New Jobs (Workflow-aware) ***
2026-04-29 22:12:57.009695 msnoise [pid 15020][DEBUG]: new_jobs --after preprocess: hpc=False, propagation is normally handled inline by worker scripts via propagate_downstream(). Running anyway for compatibility (e.g. manual re-runs, tests).
d:\pythonforsource\msnoise_stack\obspy\obspy\io\mseed\core.py:1034: UserWarning: The encoding specified in trace.stats.mseed.encoding does not match the dtype of the data.
A suitable encoding will be chosen.
warnings.warn(msg, UserWarning)
2026-04-29 22:12:57.084071 msnoise [pid 15020][INFO]: Creating CC jobs from completed preprocess jobs
2026-04-29 22:12:57.087379 msnoise [pid 15020][DEBUG]: Processing preprocess step: preprocess_1
2026-04-29 22:12:57.091972 msnoise [pid 15020][DEBUG]: Creating jobs for CC step: cc_1
2026-04-29 22:12:57.116407 msnoise [pid 15020][DEBUG]: Creating jobs for CC step: cc_2
2026-04-29 22:12:57.144235 msnoise [pid 15020][INFO]: Propagation from category "preprocess" created 0 CC job(s)
── compute_cc (cc_1 = CC and cc_2 = PCC) ───────────────────────
2026-04-29 22:12:57.145232 msnoise [pid 15020][INFO]: *** Starting: Compute CC ***
2026-04-29 22:12:57.147227 msnoise [pid 15020][DEBUG]: Checking if there are jobs to do
2026-04-29 22:12:57.226054 msnoise [INFO]: New CC batch: pair=YA.UV05.00:YA.UV06.00 n=3 group_by=day_lineage lineage=preprocess_1/cc_1
2026-04-29 22:12:57.239157 msnoise [INFO]: New CC Job: 2010-09-01 (3 pairs with 3 stations)
2026-04-29 22:12:57.335083 msnoise [pid 15020][DEBUG]: Starting slides
2026-04-29 22:12:57.337087 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T00:00:00.000000Z - 2010-09-01T00:30:00.000000Z
2026-04-29 22:12:57.433474 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T00:30:00.000000Z - 2010-09-01T01:00:00.000000Z
2026-04-29 22:12:57.452707 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T01:00:00.000000Z - 2010-09-01T01:30:00.000000Z
2026-04-29 22:12:57.471759 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T01:30:00.000000Z - 2010-09-01T02:00:00.000000Z
2026-04-29 22:12:57.489287 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T02:00:00.000000Z - 2010-09-01T02:30:00.000000Z
2026-04-29 22:12:57.507653 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T02:30:00.000000Z - 2010-09-01T03:00:00.000000Z
2026-04-29 22:12:57.526529 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T03:00:00.000000Z - 2010-09-01T03:30:00.000000Z
2026-04-29 22:12:57.543866 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T03:30:00.000000Z - 2010-09-01T04:00:00.000000Z
2026-04-29 22:12:57.562946 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T04:00:00.000000Z - 2010-09-01T04:30:00.000000Z
2026-04-29 22:12:57.582475 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T04:30:00.000000Z - 2010-09-01T05:00:00.000000Z
2026-04-29 22:12:57.600963 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T05:00:00.000000Z - 2010-09-01T05:30:00.000000Z
2026-04-29 22:12:57.619581 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T05:30:00.000000Z - 2010-09-01T06:00:00.000000Z
2026-04-29 22:12:57.637486 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T06:00:00.000000Z - 2010-09-01T06:30:00.000000Z
2026-04-29 22:12:57.659308 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T06:30:00.000000Z - 2010-09-01T07:00:00.000000Z
2026-04-29 22:12:57.684879 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T07:00:00.000000Z - 2010-09-01T07:30:00.000000Z
2026-04-29 22:12:57.704093 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T07:30:00.000000Z - 2010-09-01T08:00:00.000000Z
2026-04-29 22:12:57.722735 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T08:00:00.000000Z - 2010-09-01T08:30:00.000000Z
2026-04-29 22:12:57.740255 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T08:30:00.000000Z - 2010-09-01T09:00:00.000000Z
2026-04-29 22:12:57.759846 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T09:00:00.000000Z - 2010-09-01T09:30:00.000000Z
2026-04-29 22:12:57.779582 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T09:30:00.000000Z - 2010-09-01T10:00:00.000000Z
2026-04-29 22:12:57.797564 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T10:00:00.000000Z - 2010-09-01T10:30:00.000000Z
2026-04-29 22:12:57.815944 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T10:30:00.000000Z - 2010-09-01T11:00:00.000000Z
2026-04-29 22:12:57.833585 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T11:00:00.000000Z - 2010-09-01T11:30:00.000000Z
2026-04-29 22:12:57.851329 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T11:30:00.000000Z - 2010-09-01T12:00:00.000000Z
2026-04-29 22:12:57.869904 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T12:00:00.000000Z - 2010-09-01T12:30:00.000000Z
2026-04-29 22:12:57.887692 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T12:30:00.000000Z - 2010-09-01T13:00:00.000000Z
2026-04-29 22:12:57.905444 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T13:00:00.000000Z - 2010-09-01T13:30:00.000000Z
2026-04-29 22:12:57.922417 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T13:30:00.000000Z - 2010-09-01T14:00:00.000000Z
2026-04-29 22:12:57.940744 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T14:00:00.000000Z - 2010-09-01T14:30:00.000000Z
2026-04-29 22:12:57.958922 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T14:30:00.000000Z - 2010-09-01T15:00:00.000000Z
2026-04-29 22:12:57.977439 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T15:00:00.000000Z - 2010-09-01T15:30:00.000000Z
2026-04-29 22:12:57.994668 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T15:30:00.000000Z - 2010-09-01T16:00:00.000000Z
2026-04-29 22:12:58.013372 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T16:00:00.000000Z - 2010-09-01T16:30:00.000000Z
2026-04-29 22:12:58.031211 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T16:30:00.000000Z - 2010-09-01T17:00:00.000000Z
2026-04-29 22:12:58.049387 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T17:00:00.000000Z - 2010-09-01T17:30:00.000000Z
2026-04-29 22:12:58.068042 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T17:30:00.000000Z - 2010-09-01T18:00:00.000000Z
2026-04-29 22:12:58.086678 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T18:00:00.000000Z - 2010-09-01T18:30:00.000000Z
2026-04-29 22:12:58.105104 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T18:30:00.000000Z - 2010-09-01T19:00:00.000000Z
2026-04-29 22:12:58.123378 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T19:00:00.000000Z - 2010-09-01T19:30:00.000000Z
2026-04-29 22:12:58.141594 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T19:30:00.000000Z - 2010-09-01T20:00:00.000000Z
2026-04-29 22:12:58.161544 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T20:00:00.000000Z - 2010-09-01T20:30:00.000000Z
2026-04-29 22:12:58.181882 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T20:30:00.000000Z - 2010-09-01T21:00:00.000000Z
2026-04-29 22:12:58.200398 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T21:00:00.000000Z - 2010-09-01T21:30:00.000000Z
2026-04-29 22:12:58.219817 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T21:30:00.000000Z - 2010-09-01T22:00:00.000000Z
2026-04-29 22:12:58.237869 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T22:00:00.000000Z - 2010-09-01T22:30:00.000000Z
2026-04-29 22:12:58.255928 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T22:30:00.000000Z - 2010-09-01T23:00:00.000000Z
2026-04-29 22:12:58.274508 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T23:00:00.000000Z - 2010-09-01T23:30:00.000000Z
2026-04-29 22:12:58.292170 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T23:30:00.000000Z - 2010-09-01T23:59:59.950000Z
2026-04-29 22:12:58.495201 msnoise [INFO]: [--after cc] active cc_steps=2 active stack_steps=1
2026-04-29 22:12:58.504898 msnoise [pid 15020][DEBUG]: [--after cc] resolved 2 distinct CC→STACK lineage mappings
2026-04-29 22:12:58.506912 msnoise [INFO]: [--after cc] cc_step=cc_1 done_cc_jobs=3
2026-04-29 22:12:58.517692 msnoise [INFO]: [--after cc] cc_step=cc_2 done_cc_jobs=0
2026-04-29 22:12:58.517692 msnoise [INFO]: [--after cc] desired STACK jobs: 3
2026-04-29 22:12:58.519698 msnoise [INFO]: [--after cc] to_insert=3, to_bump=0
2026-04-29 22:12:58.523806 msnoise [INFO]: [--after cc] MOV STACK jobs: created=3, bumped_to_T=0
2026-04-29 22:12:58.542585 msnoise [INFO]: [--after cc] desired REFSTACK REF jobs: 3
2026-04-29 22:12:58.546570 msnoise [INFO]: [--after cc] REFSTACK jobs: created=3, bumped_to_T=0
2026-04-29 22:12:58.547793 msnoise [INFO]: Job Finished. It took 1.31 seconds (preprocess: 0.10 s & process 1.21 s)
2026-04-29 22:12:58.613067 msnoise [INFO]: New CC batch: pair=YA.UV05.00:YA.UV06.00 n=3 group_by=day_lineage lineage=preprocess_1/cc_2
2026-04-29 22:12:58.623513 msnoise [INFO]: New CC Job: 2010-09-01 (3 pairs with 3 stations)
2026-04-29 22:12:58.692590 msnoise [pid 15020][DEBUG]: Starting slides
2026-04-29 22:12:58.695709 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T00:00:00.000000Z - 2010-09-01T00:30:00.000000Z
2026-04-29 22:12:58.734076 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T00:30:00.000000Z - 2010-09-01T01:00:00.000000Z
2026-04-29 22:12:58.767701 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T01:00:00.000000Z - 2010-09-01T01:30:00.000000Z
2026-04-29 22:12:58.801807 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T01:30:00.000000Z - 2010-09-01T02:00:00.000000Z
2026-04-29 22:12:58.835455 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T02:00:00.000000Z - 2010-09-01T02:30:00.000000Z
2026-04-29 22:12:58.869063 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T02:30:00.000000Z - 2010-09-01T03:00:00.000000Z
2026-04-29 22:12:58.903153 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T03:00:00.000000Z - 2010-09-01T03:30:00.000000Z
2026-04-29 22:12:58.937881 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T03:30:00.000000Z - 2010-09-01T04:00:00.000000Z
2026-04-29 22:12:58.972077 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T04:00:00.000000Z - 2010-09-01T04:30:00.000000Z
2026-04-29 22:12:59.006171 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T04:30:00.000000Z - 2010-09-01T05:00:00.000000Z
2026-04-29 22:12:59.040107 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T05:00:00.000000Z - 2010-09-01T05:30:00.000000Z
2026-04-29 22:12:59.073929 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T05:30:00.000000Z - 2010-09-01T06:00:00.000000Z
2026-04-29 22:12:59.108195 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T06:00:00.000000Z - 2010-09-01T06:30:00.000000Z
2026-04-29 22:12:59.143564 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T06:30:00.000000Z - 2010-09-01T07:00:00.000000Z
2026-04-29 22:12:59.177773 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T07:00:00.000000Z - 2010-09-01T07:30:00.000000Z
2026-04-29 22:12:59.213048 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T07:30:00.000000Z - 2010-09-01T08:00:00.000000Z
2026-04-29 22:12:59.250612 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T08:00:00.000000Z - 2010-09-01T08:30:00.000000Z
2026-04-29 22:12:59.285581 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T08:30:00.000000Z - 2010-09-01T09:00:00.000000Z
2026-04-29 22:12:59.320223 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T09:00:00.000000Z - 2010-09-01T09:30:00.000000Z
2026-04-29 22:12:59.356323 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T09:30:00.000000Z - 2010-09-01T10:00:00.000000Z
2026-04-29 22:12:59.392423 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T10:00:00.000000Z - 2010-09-01T10:30:00.000000Z
2026-04-29 22:12:59.426898 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T10:30:00.000000Z - 2010-09-01T11:00:00.000000Z
2026-04-29 22:12:59.464177 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T11:00:00.000000Z - 2010-09-01T11:30:00.000000Z
2026-04-29 22:12:59.499742 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T11:30:00.000000Z - 2010-09-01T12:00:00.000000Z
2026-04-29 22:12:59.534342 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T12:00:00.000000Z - 2010-09-01T12:30:00.000000Z
2026-04-29 22:12:59.569218 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T12:30:00.000000Z - 2010-09-01T13:00:00.000000Z
2026-04-29 22:12:59.605035 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T13:00:00.000000Z - 2010-09-01T13:30:00.000000Z
2026-04-29 22:12:59.640241 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T13:30:00.000000Z - 2010-09-01T14:00:00.000000Z
2026-04-29 22:12:59.675164 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T14:00:00.000000Z - 2010-09-01T14:30:00.000000Z
2026-04-29 22:12:59.710855 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T14:30:00.000000Z - 2010-09-01T15:00:00.000000Z
2026-04-29 22:12:59.745568 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T15:00:00.000000Z - 2010-09-01T15:30:00.000000Z
2026-04-29 22:12:59.783090 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T15:30:00.000000Z - 2010-09-01T16:00:00.000000Z
2026-04-29 22:12:59.818468 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T16:00:00.000000Z - 2010-09-01T16:30:00.000000Z
2026-04-29 22:12:59.854829 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T16:30:00.000000Z - 2010-09-01T17:00:00.000000Z
2026-04-29 22:12:59.890792 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T17:00:00.000000Z - 2010-09-01T17:30:00.000000Z
2026-04-29 22:12:59.925971 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T17:30:00.000000Z - 2010-09-01T18:00:00.000000Z
2026-04-29 22:12:59.961918 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T18:00:00.000000Z - 2010-09-01T18:30:00.000000Z
2026-04-29 22:12:59.996736 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T18:30:00.000000Z - 2010-09-01T19:00:00.000000Z
2026-04-29 22:13:00.036866 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T19:00:00.000000Z - 2010-09-01T19:30:00.000000Z
2026-04-29 22:13:00.072398 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T19:30:00.000000Z - 2010-09-01T20:00:00.000000Z
2026-04-29 22:13:00.108007 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T20:00:00.000000Z - 2010-09-01T20:30:00.000000Z
2026-04-29 22:13:00.143913 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T20:30:00.000000Z - 2010-09-01T21:00:00.000000Z
2026-04-29 22:13:00.179212 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T21:00:00.000000Z - 2010-09-01T21:30:00.000000Z
2026-04-29 22:13:00.213003 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T21:30:00.000000Z - 2010-09-01T22:00:00.000000Z
2026-04-29 22:13:00.249462 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T22:00:00.000000Z - 2010-09-01T22:30:00.000000Z
2026-04-29 22:13:00.283387 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T22:30:00.000000Z - 2010-09-01T23:00:00.000000Z
2026-04-29 22:13:00.320846 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T23:00:00.000000Z - 2010-09-01T23:30:00.000000Z
2026-04-29 22:13:00.356549 msnoise [pid 15020][DEBUG]: Processing 2010-09-01T23:30:00.000000Z - 2010-09-01T23:59:59.950000Z
2026-04-29 22:13:00.542299 msnoise [INFO]: [--after cc] active cc_steps=2 active stack_steps=1
2026-04-29 22:13:00.549333 msnoise [pid 15020][DEBUG]: [--after cc] resolved 2 distinct CC→STACK lineage mappings
2026-04-29 22:13:00.550365 msnoise [INFO]: [--after cc] cc_step=cc_1 done_cc_jobs=3
2026-04-29 22:13:00.560779 msnoise [INFO]: [--after cc] cc_step=cc_2 done_cc_jobs=3
2026-04-29 22:13:00.562084 msnoise [INFO]: [--after cc] desired STACK jobs: 6
2026-04-29 22:13:00.564149 msnoise [INFO]: [--after cc] to_insert=3, to_bump=0
2026-04-29 22:13:00.567266 msnoise [INFO]: [--after cc] MOV STACK jobs: created=3, bumped_to_T=0
2026-04-29 22:13:00.586776 msnoise [INFO]: [--after cc] desired REFSTACK REF jobs: 6
2026-04-29 22:13:00.590546 msnoise [INFO]: [--after cc] REFSTACK jobs: created=3, bumped_to_T=0
2026-04-29 22:13:00.591575 msnoise [INFO]: Job Finished. It took 1.97 seconds (preprocess: 0.07 s & process 1.90 s)
2026-04-29 22:13:00.596608 msnoise [INFO]: *** Finished: Compute CC ***
── new_jobs --after cc ────────────────────────────────────────────
2026-04-29 22:13:00.597684 msnoise [INFO]: *** Starting: New Jobs (Workflow-aware) ***
2026-04-29 22:13:00.602827 msnoise [pid 15020][DEBUG]: new_jobs --after cc: hpc=False, propagation is normally handled inline by worker scripts via propagate_downstream(). Running anyway for compatibility (e.g. manual re-runs, tests).
2026-04-29 22:13:00.668035 msnoise [INFO]: [--after cc] active cc_steps=2 active stack_steps=1
2026-04-29 22:13:00.677784 msnoise [pid 15020][DEBUG]: [--after cc] resolved 2 distinct CC→STACK lineage mappings
2026-04-29 22:13:00.678784 msnoise [INFO]: [--after cc] cc_step=cc_1 done_cc_jobs=3
2026-04-29 22:13:00.690052 msnoise [INFO]: [--after cc] cc_step=cc_2 done_cc_jobs=3
2026-04-29 22:13:00.691053 msnoise [INFO]: [--after cc] desired STACK jobs: 6
2026-04-29 22:13:00.692523 msnoise [INFO]: [--after cc] to_insert=0, to_bump=0
2026-04-29 22:13:00.693618 msnoise [INFO]: [--after cc] MOV STACK jobs: created=0, bumped_to_T=0
2026-04-29 22:13:00.712005 msnoise [INFO]: [--after cc] desired REFSTACK REF jobs: 6
2026-04-29 22:13:00.714071 msnoise [INFO]: [--after cc] REFSTACK jobs: created=0, bumped_to_T=0
2026-04-29 22:13:00.714071 msnoise [INFO]: Propagation from category "cc" created/updated 0 STACK+REFSTACK job(s)
── stack_mov ──────────────────────────────────────────────────────
2026-04-29 22:13:00.715074 msnoise [pid 15020][DEBUG]: Starting the mov stack
2026-04-29 22:13:00.718657 msnoise [pid 15020][DEBUG]: Getting the next batch
2026-04-29 22:13:00.789061 msnoise [INFO]: New STACK batch: pair=YA.UV05.00:YA.UV06.00 n=1 group_by=pair_lineage lineage=preprocess_1/cc_1/filter_1/stack_1
2026-04-29 22:13:00.789061 msnoise [INFO]: New STACK Job: pair=YA.UV05.00:YA.UV06.00 n_days=1 lineage=preprocess_1/cc_1/filter_1/stack_1
2026-04-29 22:13:00.791552 msnoise [INFO]: There are STACKS jobs for some days to recompute for YA.UV05.00:YA.UV06.00
2026-04-29 22:13:00.791552 msnoise [INFO]: Processing YA.UV05.00:YA.UV06.00-ZZ MOV stack
2026-04-29 22:13:00.920188 msnoise [pid 15020][DEBUG]: [timing] load CCF (2 days): 0.13s
2026-04-29 22:13:00.973903 msnoise [pid 15020][DEBUG]: [timing] resample+compute → dr (47, 4801): 0.05s
2026-04-29 22:13:01.002729 msnoise [pid 15020][DEBUG]: [timing] rolling/resample ('1h', '1h'): 0.03s shape=(24, 4801)
2026-04-29 22:13:01.041347 msnoise [pid 15020][DEBUG]: [timing] save ('1h', '1h'): 0.03s
2026-04-29 22:13:01.052046 msnoise [INFO]: [--after refstack] No Done REF jobs found.
2026-04-29 22:13:01.118305 msnoise [pid 15020][DEBUG]: Getting the next batch
2026-04-29 22:13:01.181183 msnoise [INFO]: New STACK batch: pair=YA.UV05.00:YA.UV10.00 n=1 group_by=pair_lineage lineage=preprocess_1/cc_1/filter_1/stack_1
2026-04-29 22:13:01.181183 msnoise [INFO]: New STACK Job: pair=YA.UV05.00:YA.UV10.00 n_days=1 lineage=preprocess_1/cc_1/filter_1/stack_1
2026-04-29 22:13:01.182218 msnoise [INFO]: There are STACKS jobs for some days to recompute for YA.UV05.00:YA.UV10.00
2026-04-29 22:13:01.182218 msnoise [INFO]: Processing YA.UV05.00:YA.UV10.00-ZZ MOV stack
2026-04-29 22:13:01.200088 msnoise [pid 15020][DEBUG]: [timing] load CCF (2 days): 0.02s
2026-04-29 22:13:01.245074 msnoise [pid 15020][DEBUG]: [timing] resample+compute → dr (47, 4801): 0.04s
2026-04-29 22:13:01.271479 msnoise [pid 15020][DEBUG]: [timing] rolling/resample ('1h', '1h'): 0.03s shape=(24, 4801)
2026-04-29 22:13:01.303548 msnoise [pid 15020][DEBUG]: [timing] save ('1h', '1h'): 0.03s
2026-04-29 22:13:01.312265 msnoise [INFO]: [--after refstack] No Done REF jobs found.
2026-04-29 22:13:01.379076 msnoise [pid 15020][DEBUG]: Getting the next batch
2026-04-29 22:13:01.442524 msnoise [INFO]: New STACK batch: pair=YA.UV06.00:YA.UV10.00 n=1 group_by=pair_lineage lineage=preprocess_1/cc_1/filter_1/stack_1
2026-04-29 22:13:01.442524 msnoise [INFO]: New STACK Job: pair=YA.UV06.00:YA.UV10.00 n_days=1 lineage=preprocess_1/cc_1/filter_1/stack_1
2026-04-29 22:13:01.443977 msnoise [INFO]: There are STACKS jobs for some days to recompute for YA.UV06.00:YA.UV10.00
2026-04-29 22:13:01.443977 msnoise [INFO]: Processing YA.UV06.00:YA.UV10.00-ZZ MOV stack
2026-04-29 22:13:01.462600 msnoise [pid 15020][DEBUG]: [timing] load CCF (2 days): 0.02s
2026-04-29 22:13:01.506678 msnoise [pid 15020][DEBUG]: [timing] resample+compute → dr (47, 4801): 0.04s
2026-04-29 22:13:01.533169 msnoise [pid 15020][DEBUG]: [timing] rolling/resample ('1h', '1h'): 0.02s shape=(24, 4801)
2026-04-29 22:13:01.565478 msnoise [pid 15020][DEBUG]: [timing] save ('1h', '1h'): 0.03s
2026-04-29 22:13:01.575528 msnoise [INFO]: [--after refstack] No Done REF jobs found.
2026-04-29 22:13:01.640474 msnoise [pid 15020][DEBUG]: Getting the next batch
2026-04-29 22:13:01.704415 msnoise [INFO]: New STACK batch: pair=YA.UV05.00:YA.UV06.00 n=1 group_by=pair_lineage lineage=preprocess_1/cc_2/filter_1/stack_1
2026-04-29 22:13:01.705450 msnoise [INFO]: New STACK Job: pair=YA.UV05.00:YA.UV06.00 n_days=1 lineage=preprocess_1/cc_2/filter_1/stack_1
2026-04-29 22:13:01.706455 msnoise [INFO]: There are STACKS jobs for some days to recompute for YA.UV05.00:YA.UV06.00
2026-04-29 22:13:01.706455 msnoise [INFO]: Processing YA.UV05.00:YA.UV06.00-ZZ MOV stack
2026-04-29 22:13:01.724667 msnoise [pid 15020][DEBUG]: [timing] load CCF (2 days): 0.02s
2026-04-29 22:13:01.767667 msnoise [pid 15020][DEBUG]: [timing] resample+compute → dr (47, 4801): 0.04s
2026-04-29 22:13:01.794688 msnoise [pid 15020][DEBUG]: [timing] rolling/resample ('1h', '1h'): 0.02s shape=(24, 4801)
2026-04-29 22:13:01.827553 msnoise [pid 15020][DEBUG]: [timing] save ('1h', '1h'): 0.03s
2026-04-29 22:13:01.837137 msnoise [INFO]: [--after refstack] No Done REF jobs found.
2026-04-29 22:13:01.904556 msnoise [pid 15020][DEBUG]: Getting the next batch
2026-04-29 22:13:01.967590 msnoise [INFO]: New STACK batch: pair=YA.UV05.00:YA.UV10.00 n=1 group_by=pair_lineage lineage=preprocess_1/cc_2/filter_1/stack_1
2026-04-29 22:13:01.968622 msnoise [INFO]: New STACK Job: pair=YA.UV05.00:YA.UV10.00 n_days=1 lineage=preprocess_1/cc_2/filter_1/stack_1
2026-04-29 22:13:01.969784 msnoise [INFO]: There are STACKS jobs for some days to recompute for YA.UV05.00:YA.UV10.00
2026-04-29 22:13:01.969784 msnoise [INFO]: Processing YA.UV05.00:YA.UV10.00-ZZ MOV stack
2026-04-29 22:13:01.987469 msnoise [pid 15020][DEBUG]: [timing] load CCF (2 days): 0.02s
2026-04-29 22:13:02.030942 msnoise [pid 15020][DEBUG]: [timing] resample+compute → dr (47, 4801): 0.04s
2026-04-29 22:13:02.057713 msnoise [pid 15020][DEBUG]: [timing] rolling/resample ('1h', '1h'): 0.02s shape=(24, 4801)
2026-04-29 22:13:02.088734 msnoise [pid 15020][DEBUG]: [timing] save ('1h', '1h'): 0.03s
2026-04-29 22:13:02.098648 msnoise [INFO]: [--after refstack] No Done REF jobs found.
2026-04-29 22:13:02.168658 msnoise [pid 15020][DEBUG]: Getting the next batch
2026-04-29 22:13:02.232956 msnoise [INFO]: New STACK batch: pair=YA.UV06.00:YA.UV10.00 n=1 group_by=pair_lineage lineage=preprocess_1/cc_2/filter_1/stack_1
2026-04-29 22:13:02.232956 msnoise [INFO]: New STACK Job: pair=YA.UV06.00:YA.UV10.00 n_days=1 lineage=preprocess_1/cc_2/filter_1/stack_1
2026-04-29 22:13:02.233962 msnoise [INFO]: There are STACKS jobs for some days to recompute for YA.UV06.00:YA.UV10.00
2026-04-29 22:13:02.234976 msnoise [INFO]: Processing YA.UV06.00:YA.UV10.00-ZZ MOV stack
2026-04-29 22:13:02.251703 msnoise [pid 15020][DEBUG]: [timing] load CCF (2 days): 0.02s
2026-04-29 22:13:02.295342 msnoise [pid 15020][DEBUG]: [timing] resample+compute → dr (47, 4801): 0.04s
2026-04-29 22:13:02.321462 msnoise [pid 15020][DEBUG]: [timing] rolling/resample ('1h', '1h'): 0.02s shape=(24, 4801)
2026-04-29 22:13:02.353053 msnoise [pid 15020][DEBUG]: [timing] save ('1h', '1h'): 0.03s
2026-04-29 22:13:02.363285 msnoise [INFO]: [--after refstack] No Done REF jobs found.
── new_jobs --after stack ─────────────────────────────────────────
2026-04-29 22:13:02.430432 msnoise [INFO]: *** Starting: New Jobs (Workflow-aware) ***
2026-04-29 22:13:02.435113 msnoise [pid 15020][DEBUG]: new_jobs --after stack: hpc=False, propagation is normally handled inline by worker scripts via propagate_downstream(). Running anyway for compatibility (e.g. manual re-runs, tests).
2026-04-29 22:13:02.500622 msnoise [INFO]: [--after refstack] No Done REF jobs found.
2026-04-29 22:13:02.563482 msnoise [INFO]: Propagation from category "stack" created 0 MWCS/stretching/wavelet job(s)
Done.
11 · Gather MSNoiseResult Objects
[13]:
db = connect()
# Lineage for CC branch: preprocess_1 / cc_1 / filter_1 / stack_1
result_cc = MSNoiseResult.from_ids(
db,
preprocess=1,
cc=cc1_sn, # cc_1
filter=1,
stack=1,
)
print("CC lineage:", result_cc.lineage_names)
# Lineage for PCC branch: preprocess_1 / cc_2 / filter_1 / stack_1
result_pcc = MSNoiseResult.from_ids(
db,
preprocess=1,
cc=cc2_sn, # cc_2
filter=1,
stack=1,
)
print("PCC lineage:", result_pcc.lineage_names)
CC lineage: ['preprocess_1', 'cc_1', 'filter_1', 'stack_1']
PCC lineage: ['preprocess_1', 'cc_2', 'filter_1', 'stack_1']
12 · Compare Stacked CCFs
[14]:
# Retrieve all stacked CCFs for both lineages
ccfs_cc = result_cc.get_ccf()
ccfs_pcc = result_pcc.get_ccf()
print(f"CC result: {len(ccfs_cc)} CCF(s) found")
print(f"PCC result: {len(ccfs_pcc)} CCF(s) found")
for key in sorted(ccfs_cc):
pair, comp, ms = key
print(f" {pair} {comp} {ms[0]}–{ms[1]}")
CC result: 3 CCF(s) found
PCC result: 3 CCF(s) found
YA.UV05.00:YA.UV06.00 ZZ 1h–1h
YA.UV05.00:YA.UV10.00 ZZ 1h–1h
YA.UV06.00:YA.UV10.00 ZZ 1h–1h
13 · Plot: CC vs PCC per Pair
[15]:
fig_dir = os.path.join(WORK_DIR, "figures")
os.makedirs(fig_dir, exist_ok=True)
common_keys = sorted(set(ccfs_cc) & set(ccfs_pcc))
if not common_keys:
print("No common keys found between CC and PCC results — check that "
"the pipeline completed successfully.")
else:
n = len(common_keys)
fig, axes = plt.subplots(n, 2, figsize=(14, 3.5 * n), squeeze=False)
fig.suptitle(
f"Stacked CCF comparison: CC (left) vs PCC (right)\n"
f"Filter {FREQMIN}–{FREQMAX} Hz | {STARTDATE}",
fontsize=13, y=1.01,
)
for row, key in enumerate(common_keys):
pair, comp, ms = key
da_cc = ccfs_cc[key]
da_pcc = ccfs_pcc[key]
# Time axis (seconds)
taxis = da_cc.coords["taxis"].values if "taxis" in da_cc.coords \
else np.linspace(-MAXLAG, MAXLAG, da_cc.shape[-1])
# Stack over the time dimension if multiple days present
ccf_cc = float(da_cc.mean()) if da_cc.ndim == 0 else da_cc.values
ccf_pcc = float(da_pcc.mean()) if da_pcc.ndim == 0 else da_pcc.values
if ccf_cc.ndim > 1:
ccf_cc = ccf_cc.mean(axis=0)
if ccf_pcc.ndim > 1:
ccf_pcc = ccf_pcc.mean(axis=0)
label = f"{pair} {comp}"
# ── CC panel ──────────────────────────────────────────────────────
ax = axes[row, 0]
ax.plot(taxis, ccf_cc, color="#1f77b4", lw=0.9)
ax.axvline(0, color="k", lw=0.6, ls="--", alpha=0.4)
ax.set_title(f"{label} — CC")
ax.set_xlabel("Lag (s)")
ax.set_ylabel("Amplitude")
ax.set_xlim(-MAXLAG, MAXLAG)
# ── PCC panel ─────────────────────────────────────────────────────
ax = axes[row, 1]
ax.plot(taxis, ccf_pcc, color="#d62728", lw=0.9)
ax.axvline(0, color="k", lw=0.6, ls="--", alpha=0.4)
ax.set_title(f"{label} — PCC")
ax.set_xlabel("Lag (s)")
ax.set_ylabel("Amplitude")
ax.set_xlim(-MAXLAG, MAXLAG)
plt.tight_layout()
plt.show()
[16]:
import numpy as np
A = np.fft.fft(ccf_cc)
f = np.fft.fftfreq(len(A), d=1/20.)[:len(A)//2]
A /= A.max()
B = np.fft.fft(ccf_pcc)
B /= B.max()
plt.figure(figsize=(15,5))
plt.plot(f, abs(A[:len(A)//2]))
plt.plot(f, abs(B[:len(B)//2]))
# plt.ylim(0,0.15)
plt.xlim(0,5)
[16]:
(0.0, 5.0)
14 · Overlay: CC vs PCC on the same axes (one panel per pair)
[17]:
if common_keys:
fig2, axes2 = plt.subplots(1, len(common_keys),
figsize=(6 * len(common_keys), 4),
squeeze=False)
fig2.suptitle(
f"CC (blue) vs PCC (red) | {FREQMIN}–{FREQMAX} Hz | {STARTDATE}",
fontsize=12,
)
for col, key in enumerate(common_keys):
pair, comp, ms = key
da_cc = ccfs_cc[key]
da_pcc = ccfs_pcc[key]
taxis = da_cc.coords["taxis"].values if "taxis" in da_cc.coords \
else np.linspace(-MAXLAG, MAXLAG, da_cc.values.shape[-1])
ccf_cc = da_cc.values
ccf_pcc = da_pcc.values
if ccf_cc.ndim > 1: ccf_cc = ccf_cc.mean(axis=0)
if ccf_pcc.ndim > 1: ccf_pcc = ccf_pcc.mean(axis=0)
# Normalise to ABSMAX for visual overlay
def _norm(x):
m = np.abs(x).max()
return x / m if m > 0 else x
ax = axes2[0, col]
ax.plot(taxis, _norm(ccf_cc), color="#1f77b4", lw=1.0,
label="CC", alpha=0.85)
ax.plot(taxis, _norm(ccf_pcc), color="#d62728", lw=1.0,
label="PCC", alpha=0.85)
ax.axvline(0, color="k", lw=0.6, ls="--", alpha=0.3)
ax.set_title(f"{pair} {comp}")
ax.set_xlabel("Lag (s)")
ax.set_ylabel("Normalised amplitude")
ax.set_xlim(-MAXLAG, MAXLAG)
ax.legend(loc="upper right", fontsize=9)
plt.tight_layout()
plt.show()
15 · Summary Table
[18]:
import pandas as pd
rows = []
for key in common_keys:
pair, comp, ms = key
cc_arr = ccfs_cc[key].values
pcc_arr = ccfs_pcc[key].values
if cc_arr.ndim > 1: cc_arr = cc_arr.mean(axis=0)
if pcc_arr.ndim > 1: pcc_arr = pcc_arr.mean(axis=0)
rows.append({
"Pair": pair,
"Component": comp,
"Mov. stack": f"{ms[0]}–{ms[1]}",
"CC peak": f"{np.abs(cc_arr).max():.4g}",
"PCC peak": f"{np.abs(pcc_arr).max():.4g}",
"CC lag@peak (s)": f"{np.where(np.abs(cc_arr) == np.abs(cc_arr).max())[0][0] - len(cc_arr)//2:.1f}",
"PCC lag@peak (s)": f"{np.where(np.abs(pcc_arr) == np.abs(pcc_arr).max())[0][0] - len(pcc_arr)//2:.1f}",
})
df = pd.DataFrame(rows)
print(df.to_string(index=False))
Pair Component Mov. stack CC peak PCC peak CC lag@peak (s) PCC lag@peak (s)
YA.UV05.00:YA.UV06.00 ZZ 1h–1h 3.978e-07 0.02806 -43.0 -43.0
YA.UV05.00:YA.UV10.00 ZZ 1h–1h 3.862e-07 0.02519 29.0 29.0
YA.UV06.00:YA.UV10.00 ZZ 1h–1h 3.386e-07 0.02356 -22.0 -22.0
Notes
Why PCC output looks different from CC
CC retains amplitude information: large-energy windows (earthquakes, transients) dominate the stack even after winsorising. The CCF contains the full filter-band content weighted by window energy.
PCC (v=2) projects every sample of the analytic signal onto the unit circle before correlating — amplitude is discarded per-sample. The output is bounded to [−1, 1] by construction. The PCC branch in s03:
bandpasses to the filter_N frequencies so each filter is distinct,
optionally whitens within the band (controlled by
whiteningconfig) to give a broadband phase signal (“spectrally whitened PCC”),then calls
pcc_xcorrwhich computes AmpNorm + FFT cross-correlation.
Winsorising is disabled for
cc_2: it clips amplitude extremes, which is meaningless after AmpNorm has already equalised every sample to unit amplitude.Whitening = “N” variant: set
whiteningto"N"oncc_2for classic (non-whitened) PCC. The result will be dominated by the most energetic frequency within the preprocessing bandpass — often visually narrower than CC. The default (whitening="A") gives a more spectrally balanced PCC that compares directly to CC.
Adjusting the comparison
Change
FREQMIN/FREQMAXin the User Settings cell and re-run from §6 onwards — both lineages sharefilter_1, so re-running the stack step is sufficient (no need to redo preprocessing or CC).The
result_ccandresult_pccobjects expose the full MSNoiseResult API: call.get_mwcs(),.get_dvv(), etc. after running the downstream steps on each lineage.