Release date: 20 March 2015
Release type: major
8 months after the last bugfix release (MSNoise 1.2.5), and 17 months after the last major release (MSNoise 1.2) we are proud to announce the new MSNoise 1.3. It is a major release, with a massive amount of work since the last release: in GitHub numbers , it’s over 100 commits and about 3500 new lines of code and documentation added ! MSNoise 1.3 introduces a brand new way of executing the workflow. The workflow in itself doesn’t change, so experienced users as well as new users reading the SRL publication will find their way easily!
MSNoise is now a Python Package, allowing a single (and easy) install for
all your projects and/or all users using
pip. The new top-level
command contains all the steps of the workflow, plus new additions, as the very
reset command to easily mark all jobs “T”odo. The
command group which includes seven plots, all directly callable from the command
line, without needing to hack/edit the source codes. About hacking: MSNoise has
now a proper documented API which allows pythonistas to write their own plots,
computation steps, …, while interacting with the database and the data
archive! The “dynamic time lag” allows to use parts of the coda that is
dependent from the interstation distance (provided station coordinates are
Finally, MSNoise is now tested and automatically checked by Travis-CI!
This version has benefited from outputs/ideas/pull requests/questions from several users:
Oscar Alberto Castro Artola
Kasper van Wijk
Kohtaro R. Araragi
all participants to the 2014 Pre-AGU MSNoise workshop
all others (don’t be mad :-) )
Thanks to all for using MSNoise, and please, let us know why/how you use it (and please cite it!)!
Thomas Lecocq & Corentin Caudron
PS: if you use MSNoise for your research and prepare publications, please consider citing it:
Lecocq, T., C. Caudron, et F. Brenguier (2014), MSNoise, a Python Package for Monitoring Seismic Velocity Changes Using Ambient Seismic Noise, Seismological Research Letters, 85(3), 715‑726, doi:10.1785/0220130073.
This is probably the most important change since the original release of MSNoise 1.0 (August 2013), it represents a massive amount of work since the last release (1.2.5 in June 2014), and is probably the most needed by users! In GitHub numbers , it’s over 100 commits and about 3500 new lines of code (and of documentation!) added !
In practice, what does change ?
MSNoise is installable using
easy_install, soon using
MSNoise is installed in the common “site-package/” folder of one’s python install.
Once installed, it is available for all users, all projects.
It allows updating MSNoise for all projects at once.
It removes all python files from project folders, which is much cleaner.
MSNoise being installed in the standard lib directories means it shouldn’t
write or output anything in those folders. To facilitate the launch of commnands
a new top level
msnoise command has been created, and should be available
right after installing.
Users of MSNoise will have to change the way they call the steps, i.e.:
All the commands are visible using the
msnoise --help Usage: msnoise-script.py [OPTIONS] COMMAND [ARGS]... Options: -t, --threads INTEGER Number of threads to use (only affects modules that are designed to do parallel processing) --help Show this message and exit. Commands: bugreport This command launches the Bug Report script. compute_cc Computes the CC jobs (based on the "New Jobs"... compute_dtt Computes the dt/t jobs based on the new MWCS... compute_mwcs Computes the MWCS based on the new stacked... compute_stretching [experimental] Computes the stretching based... config This command launches the Configurator. info Outputs general information about the current... install This command launches the installer. ipython Launches an ipython notebook in the current... new_jobs Determines if new CC jobs are to be defined plot Top level command to trigger different plots populate Rapidly scan the archive filenames and find... reset Resets the job to "T"odo. scan_archive Scan the archive and insert into the Data... stack Stacks the [REF] and/or [MOV] windows test Runs the test suite, should be executed in an... upgrade_db Upgrade the database from pre-1.3 to MSNoise...
The parameters/arguments of each command are explained using its own
, for example:
msnoise reset --help Usage: msnoise-script.py reset [OPTIONS] JOBTYPE Resets the job to "T"odo. ARG is [CC] or [DTT]. By default only resets jobs "I"n progress. --all resets all jobs, whatever the flag value Options: -a, --all Reset all jobs --help Show this message and exit.
The description of each step has been updated in the documentation.
As explained above,
msnoise is a top level command available in your
command prompt. MSNoise 1.3 includes several plots which are available using the
msnoise plot command. See Plotting to view all plots!
All the available plots are listed using the
msnoise plot --help Usage: msnoise-script.py plot [OPTIONS] COMMAND [ARGS]... Top level command to trigger different plots Options: --help Show this message and exit. Commands: ccftime Plots the dv/v (parses the dt/t results) data_availability Plots the Data Availability vs time distance Plots the REFs of all pairs vs distance dvv Plots the dv/v (parses the dt/t results) interferogram Plots the interferogram between sta1 and sta2... mwcs Plots the mwcs results between sta1 and sta2... station_map Plots the station map (very basic)
Same as above, sub-commands have their own
msnoise plot interferogram --help Usage: [OPTIONS] STA1 STA2 [EXTRA_ARGS]... Plots the interferogram between sta1 and sta2 (parses the CCFs) STA1 and STA2 must be provided with this format: NET.STA ! Options: -f, --filterid INTEGER Filter ID -c, --comp TEXT Components (ZZ, ZR,...) -m, --mov_stack INTEGER Mov Stack to read from disk -s, --show BOOLEAN Show interactively? -o, --outfile TEXT Output filename (?=auto) -r, --refilter TEXT Refilter CCFs before plotting (e.g. 4:8 for filtering CCFs between 4.0 and 8.0 Hz. This will update the plot title. --help Show this message and exit.
As before, the dt/t is determined as the slope of the delays vs time lags.
the slope is calculated a weighted linear regression (WLS) through selected
The selection of points is first based on the time lag criteria.
The minimum time lag can either be defined absolutely or dynamically.
dtt_lag is set to “dynamic” in the database, the inter-station
distance is used to determine the minimum time lag. This lag is calculated from
the distance and a velocity configured (
dtt_v). The velocity is determined
by the user so that the minlag doesn’t include the ballistic waves. For example
if ballistic waves are visible with a velocity of 2 km/s, one could configure
This way, if stations are located 15 km apart, the minimum lag time will be
set to 15 s. The
dtt_width determines the width of the lag window used. A
value of 30.0 means the process will use time lags between 15 and 45 s in the
example above, on both sides if configured (
dtt_sides), or only causal or
acausal parts of the CCF.
It seems obvious that these parameters are frequency-dependent, but they are currently common for all filters !
New parameters have been added to the configuration:
dtt_lag: How is the lag window defined dynamic/[static] (default=static)
dtt_lag=dynamic: what velocity to use to avoid ballistic wave (in km/s) (default=1)
dtt_lag=static: min lag time (in seconds) (default=5)
dtt_width: Width of the time lag window (in seconds) (default=30)
dtt_sides: Which sides to use [both]/left/right (default=both)
dtt_mincoh: Minimum coherence on dt measurement, MWCS points with values lower than that will not be used in the WLS, [0:1] (default=0.65)
dtt_maxerr: Maximum error on dt measurement, MWCS points with values larger than that will not be used in the WLS [0:1] (default=0.1)
dtt_maxdt: Maximum dt values, MWCS points with values larger than that will not be used in the WLS (in seconds) (default=0.1)
The description of the Compute dt/t step in the workflow.
Some improvements to the maths have been done for MSNoise 1.3:
whiten: the symmetric hermitian was not properly defined and could lead to a 1 sample shift in the negative frequencies.
compute_cc: it is now possible to define an overlap of the windows
-1now computes the 1-bit normalization of the trace. Reminder:
0: no normalization,
N: N*rms clipping.
mwcs: the tapering of the windowed CCF has been improved in order to optimize the calculation for the center of the window.
Improvements in terms of performances have also been done for MSNoise 1.3:
new_jobs: the procedure has been completely rewritten and should be a lot faster, certainly for large (to very-large) archives and/or number of days.
keep_all: if set to
Y(=True) in the config, all CCF are now stored in a single HDF5 file, which makes it much nicer to backup/transfer/delete.
compute_cc: if only ZZ components are to be computed, the whitened windows are pre-computed, which makes the process faster. This could lead to memory issues if the job contains a lot of stations, a lot of filters are configured and a large number of windows.
compute_mwcs: The procedure updates the jobs all at once, which brings a big gain in transaction time.
database_tools.py has been renamed to
api.py and all the
functions are now documented (in MSNoise API) so they can be used from the
console or from custom scripts.
msnoise ipython command, one triggers the start of an IPython
notebook in the current project folder. Once in a new notebook, one could
from msnoise.api import connect, get_config session = connect() print get_config(session, "mov_stack")
to get the current configuration of the
mov_stack parameter ! Enjoy Hacking!
UnitTests are now defined for some (not all, yet) MSNoise functions, and most of
the workflow is tested automatically by launching
msnoise test in a new
This will allow contributors to easily know the code they want to be merged in the next versions of MSNoise doesn’t break anything. Travis-CI runs automatically on every push or pull request made on GitHub.
Some users will want to keep their current project without recomputing everything. This requires:
adding a few configuration parameters to the database
modifying the structure of the
Running the following command will do both parts for MySQL and only the first part for SQLite:
The second part is a little different if you are using SQLite as it
can’t be done automatically. This is because SQLite doesn’t support “ALTER”
commands. Ultimately we want the
jobs.type to be renamed to
jobs.jobtype. You will have to do this operation manually:
Open SQLite database browser (SQLiteManager extension for Firefox, for example)
Open the msnoise.sqlite file
Select the jobs table
typefield and rename it to
Ignore the warnings (it should work, although it could fail!)
Close the database
msnoise command accepts the
-t INTEGER argument to launch a number of threads in parallel, it currently
only works with
msnoise -t 4 scan_archive will run the
scan on four folders in parallel. For the other steps, one has still to run
multiple commands in a console. This should change in the future.