MSNoise API¶

msnoise.api.get_logger(name, loglevel=None, with_pid=False)¶: Returns the current configured logger or configure a new one.

msnoise.api.get_engine(inifile=None)¶

Returns the a SQLAlchemy Engine

Parameters: inifile (str) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini
Return type: sqlalchemy.engine.Engine
Returns: An Engine Object

msnoise.api.connect(inifile=None)¶

Establishes a connection to the database and returns a Session object.

Parameters: inifile (string) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini
Return type: sqlalchemy.orm.session.Session
Returns: A Session object, needed for many of the other API methods.

msnoise.api.create_database_inifile(tech, hostname, database, username, password, prefix='')¶

Creates the db.ini file based on supplied parameters.

Parameters

tech (int) – The database technology used: 1=sqlite 2=mysql
hostname (string) – The hostname of the server (if tech=2) or the name of the sqlite file if tech=1)
database (string) – The database name
username (string) – The user name
prefix (string) – The prefix to use for all tables
password (string) – The password of user

Returns

None

msnoise.api.read_db_inifile(inifile=None)¶

Reads the parameters from the db.ini file.

Parameters: inifile (string) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini
Return type: tuple
Returns: tech, hostname, database, username, password

msnoise.api.get_config(session, name=None, isbool=False, plugin=None)¶

Get the value of one or all config bits from the database.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
name (str) – The name of the config bit to get. If omitted, a dictionnary with all config items will be returned
isbool (bool) – if True, returns True/False for config name. Defaults to False
plugin (str) – if provided, gives the name of the Plugin config to use. E.g. if “Amazing” is provided, MSNoise will try to load the “AmazingConfig” entry point. See Extending MSNoise with Plugins for details.

Return type

str, bool or dict

Returns

the value for name or a dict of all config values

msnoise.api.update_config(session, name, value, plugin=None)¶

Update one config bit in the database.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
name (str) – The name of the config bit to set.
value (str) – The value of parameter name. Can also be NULL if you don’t want to use this particular parameter.
plugin (str) – if provided, gives the name of the Plugin config to use. E.g. if “Amazing” is provided, MSNoise will try to load the “AmazingConfig” entry point. See Extending MSNoise with Plugins for details.

msnoise.api.get_params(session)¶

Get config parameters from the database.

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Returns: a Param class containing the parameters

msnoise.api.get_filters(session, all=False)¶

Get Filters from the database.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
all (bool) – Returns all filters from the database if True, or only filters where used = 1 if False (default)

Return type

list of Filter

Returns

a list of Filter

msnoise.api.update_filter(session, ref, low, mwcs_low, high, mwcs_high, rms_threshold, mwcs_wlen, mwcs_step, used)¶

Updates or Insert a new Filter in the database.

See also

msnoise.msnoise_table_def.declare_tables.Station

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
net (str) – The network code of the Station
sta (str) – The station code
X (float) – The X coordinate of the station
Y (float) – The Y coordinate of the station
altitude (float) – The altitude of the station
coordinates (str) – The coordinates system. “DEG” is WGS84 latitude/ longitude in degrees. “UTM” is expressed in meters.
instrument (str) – The instrument code, useful with PAZ correction
used (bool) – Whether this station must be used in the computations.

msnoise.api.get_station_pairs(session, used=None, net=None)¶

Returns an iterator over all possible station pairs. If auto-correlation is configured in the database, returns N*N pairs, otherwise returns N*(N-1)/2 pairs.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
used (bool, int) – Select only stations marked used if False (default) or all stations present in the database if True
net (str) – Network code to filter for the pairs.

Return type

iterable

Returns

An iterable of Station object pairs

msnoise.api.get_interstation_distance(station1, station2, coordinates='DEG')¶

Returns the distance in km between station1 and station2.

Warning

Currently the stations coordinates system have to be the same!

Parameters

station1 (Station) – A Station object
station2 (Station) – A Station object
coordinates (str) – The coordinates system. “DEG” is WGS84 latitude/ longitude in degrees. “UTM” is expressed in meters.

Return type

float

Returns

The interstation distance in km

msnoise.api.update_data_availability(session, net, sta, comp, path, file, starttime, endtime, data_duration, gaps_duration, samplerate)¶

Updates a DataAvailability object in the database

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
net (str) – The network code of the Station
sta (str) – The station code
comp (str) – The component (channel)
path (str) – The full path to the folder containing the file
file (str) – The name of the file
starttime (datetime.datetime) – Start time of the file
endtime (datetime.datetime) – End time of the file
data_duration (float) – Cumulative duration of available data in the file
gaps_duration (float) – Cumulative duration of gaps in the file
samplerate (float) – Sample rate of the data in the file (in Hz)

msnoise.api.get_new_files(session)¶

Returns the files marked “N”ew or “M”odified in the database

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type: list
Returns: list of DataAvailability

msnoise.api.get_data_availability(session, net=None, sta=None, comp=None, starttime=None, endtime=None)¶

Returns the DataAvailability objects for specific net, sta, starttime or endtime

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
net (str) – Network code
sta (str) – Station code
starttime (datetime.datetime, datetime.date) – Start time of the search
endtime (datetime.datetime, datetime.date) – End time of the search

Return type

list

Returns

list of DataAvailability

msnoise.api.mark_data_availability(session, net, sta, flag)¶

Updates the flag of all DataAvailability objects matching net.sta in the database

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
net (str) – Network code
sta (str) – Station code
flag (str) – Status of the DataAvailability object: New, Modified or Archive. Values accepted are {‘N’, ‘M’, ‘A’}

msnoise.api.count_data_availability_flags(session)¶

Count the number of DataAvailability, grouped by flag

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type: list
Returns: list of [count, flag] pairs

msnoise.api.update_job(session, day, pair, jobtype, flag, commit=True, returnjob=True, ref=None)¶

Updates or Inserts a new Job in the database.

Parameters

day (str) – The day in YYYY-MM-DD format
pair (str) – the name of the pair (EXAMPLE?)
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.
commit (bool) – Whether to directly commit (True, default) or not (False)
returnjob (bool) – Return the modified/inserted Job (True, default) or not (False)

Return type

Job or None

Returns

If returnjob is True, returns the modified/inserted Job.

msnoise.api.massive_insert_job(jobs)¶

Routine to use a low level function to insert much faster a list of Job. This method uses the Engine directly, no need to pass a Session object.

Parameters: jobs (list) – a list of Job to insert.

msnoise.api.massive_update_job(session, jobs, flag='D')¶

Routine to use a low level function to update much faster a list of Job. This method uses the Job.ref which is unique.

Parameters

jobs (list) – a list of Job to update.
flag (str) – The destination flag.

msnoise.api.is_next_job(session, flag='T', jobtype='CC')¶

Are there any Job in the database, with flag=`flag` and jobtype=`type`

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.

Return type

bool

Returns

True if at least one Job matches, False otherwise.

msnoise.api.get_next_job(session, flag='T', jobtype='CC')¶

Get the next Job in the database, with flag=`flag` and jobtype=`jobtype`. Jobs of the same type are grouped per day. This function also sets the flag of all selected Jobs to “I”n progress.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.

Return type

list

Returns

list of Job

msnoise.api.is_dtt_next_job(session, flag='T', jobtype='DTT', ref=False)¶

Are there any DTT Job in the database, with flag=`flag` and jobtype=`jobtype`. If ref is provided, checks if a DTT “REF” job is present.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.
ref (bool) – Whether to check for a REF job (True) or not (False, default)

Return type

bool

Returns

True if at least one Job matches, False otherwise.

msnoise.api.get_dtt_next_job(session, flag='T', jobtype='DTT')¶

Get the next DTT Job in the database, with flag=`flag` and jobtype=`jobtype`. Jobs are then grouped per station pair. This function also sets the flag of all selected Jobs to “I”n progress.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.

Return type

tuple

Returns

(pairs, days, refs): List of station pair names - Days of the next DTT jobs - Job IDs (for later being able to update their flag).

msnoise.api.reset_jobs(session, jobtype, alljobs=False, rule=None)¶

Sets the flag of all jobtype Jobs to “T”odo.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
alljobs (bool) – If True, resets all jobs. If False (default), only resets jobs “I”n progress.

msnoise.api.reset_dtt_jobs(session, pair)¶

Sets the flag of all DTT Jobs of one pair to “T”odo.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
pair (str) – The pair to update

msnoise.api.get_job_types(session, jobtype='CC')¶

Count the number of Jobs of a specific type, grouped by flag.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?

Return type

list

Returns

list of [count, flag] pairs

msnoise.api.get_jobs_by_lastmod(session, jobtype='CC', lastmod=datetime.datetime(2020, 2, 25, 17, 25, 15, 593597))¶

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
lastmod (datetime.datetime) – Jobs’ modification time

Return type

list

Returns

list of Job objects.

msnoise.api.export_allcorr(session, ccfid, data)¶

msnoise.api.export_allcorr2(session, ccfid, data)¶

msnoise.api.add_corr(session, station1, station2, filterid, date, time, duration, components, CF, sampling_rate, day=False, ncorr=0, params=None)¶

Adds a CCF to the data archive on disk.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
station1 (str) – The name of station 1 (formatted NET.STA)
station2 (str) – The name of station 2 (formatted NET.STA)
filterid (int) – The ID (ref) of the filter
date (datetime.date or str) – The date of the CCF
time (datetime.time or str) – The time of the CCF
duration (float) – The total duration of the exported CCF
components (str) – The name of the components used (ZZ, ZR, …)
sampling_rate (float) – The sampling rate of the exported CCF
day (bool) – Whether this function is called to export a daily stack (True) or each CCF (when keep_all parameter is set to True in the configuration). Defaults to True.
ncorr (int) – Number of CCF that have been stacked for this CCF.
params (dict) – A dictionnary of MSNoise config parameters as returned by get_params().

msnoise.api.export_sac(db, filename, pair, components, filterid, corr, ncorr=0, sac_format=None, maxlag=None, cc_sampling_rate=None, params=None)¶

msnoise.api.export_mseed(db, filename, pair, components, filterid, corr, ncorr=0, maxlag=None, cc_sampling_rate=None, params=None)¶

msnoise.api.stack(data, stack_method='linear', pws_timegate=10.0, pws_power=2, goal_sampling_rate=20.0)¶

Parameters

data (numpy.ndarray) – the data to stack, each row being one CCF
stack_method (str) – either linear: average of all CCF or pws to compute the phase weigthed stack. If pws is selected, the function expects the pws_timegate and pws_power.
pws_timegate (float) – PWS time gate in seconds. Width of the smoothing window to convolve with the PWS spectrum.
pws_power (float) – Power of the PWS weights to be applied to the CCF stack.
goal_sampling_rate (float) – Sampling rate of the CCF array submitted

Return type

numpy.array

Returns

the stacked CCF.

msnoise.api.get_results(session, station1, station2, filterid, components, dates, mov_stack=1, format='stack', params=None)¶

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
station1 (str) – The name of station 1 (formatted NET_STA)
station2 (str) – The name of station 2 (formatted NET_STA)
filterid (int) – The ID (ref) of the filter
components (str) – The name of the components used (ZZ, ZR, …)
dates (list) – List of TODO datetime.datetime
mov_stack (int) – Moving window stack.
format (str) – Either stack: the data will be stacked according to the parameters passed with params or matrix: to get a 2D array of CCF.
params (dict) – A dictionnary of MSNoise config parameters as returned by get_params().

Return type

numpy.ndarray

Returns

Either a 1D CCF (if format is stack or a 2D array (if format= matrix).

msnoise.api.get_results_all(session, station1, station2, filterid, components, dates)¶

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
station1 (str) – The name of station 1 (formatted NET_STA)
station2 (str) – The name of station 2 (formatted NET_STA)
filterid (int) – The ID (ref) of the filter
components (str) – The name of the components used (ZZ, ZR, …)
dates (list) – List of TODO datetime.datetime

Return type

pandas.DataFrame

Returns

All CCF results in a pandas.DataFrame, where the index is the time of the CCF and the columns are the times in the coda.

msnoise.api.get_maxlag_samples(session)¶

Returns the length of the CC functions. Gets the maxlag and sampling rate from the database.

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type: int
Returns: the length of the CCF in samples

msnoise.api.get_t_axis(session)¶

Returns the time axis (in seconds) of the CC functions. Gets the maxlag from the database and uses get_maxlag_samples function.

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type: numpy.array
Returns: the time axis in seconds

msnoise.api.get_components_to_compute(session, plugin=None)¶

Returns the components configured in the database.

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type: list of str
Returns: a list of components to compute

msnoise.api.get_components_to_compute_single_station(session, plugin=None)¶

Returns the components configured in the database.

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type: list of str
Returns: a list of components to compute

msnoise.api.build_ref_datelist(session)¶

Creates a date array for the REF. The returned tuple contains a start and an end date, and a list of individual dates between the two.

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type: tuple
Returns: (start, end, datelist)

msnoise.api.build_movstack_datelist(session)¶

Creates a date array for the analyse period. The returned tuple contains a start and an end date, and a list of individual dates between the two.

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type: tuple
Returns: (start, end, datelist)

msnoise.api.updated_days_for_dates(session, date1, date2, pair, jobtype='CC', interval=datetime.timedelta(days=1), returndays=False)¶

Determines if any Job of jobtype=`jobtype` and for pair=`pair`, concerning a date between date1 and date2 has been modified in the last interval=`interval`.

Parameters

session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
date1 (datetime.datetime) – Beginning of the period of interest
date2 (datetime.datetime) – End of the period of interest
pair (str) – Pair of interest
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
interval (datetime.timedelta) – Interval of time before now to search for updated days
returndays (bool) – Whether to return a list of days (True) or not (False, default)

Return type

list or bool

Returns

List of days if returndays is True, only “True” if not. (not clear!)

msnoise.api.azimuth(coordinates, x0, y0, x1, y1)¶

Returns the azimuth between two coordinate sets.

Parameters

coordinates (str) – {‘DEG’, ‘UTM’, ‘MIX’}
x0 (float) – X coordinate of station 1
y0 (float) – Y coordinate of station 1
x1 (float) – X coordinate of station 2
y1 (float) – Y coordinate of station 2

Return type

float

Returns

The azimuth in degrees

msnoise.api.nextpow2(x)¶

Returns the next power of 2 of x.

Parameters: x (int) – any value
Return type: int
Returns: the next power of 2 of x

msnoise.api.check_and_phase_shift(trace, taper_length=20.0)¶

msnoise.api.getGaps(stream, min_gap=None, max_gap=None)¶

msnoise.api.make_same_length(st)¶: This function takes a stream of equal sampling rate and makes sure that all channels have the same length and the same gaps.

msnoise.api.clean_scipy_cache()¶: This functions wraps all destroy scipy cache at once. It is a workaround to the memory leak induced by the “caching” functions in scipy fft.

msnoise.api.preload_instrument_responses(session)¶

This function preloads all instrument responses from response_format and stores the seed ids, start and end dates, and paz for every channel in a DataFrame.

Warning

This function only works for response_format being “inventory” or “dataless”.

Parameters: session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type: pandas.DataFrame
Returns: A table containing all channels with the time of operation and poles and zeros.