MSNoise API
- msnoise.api.get_logger(name, loglevel=None, with_pid=False)
Returns the current configured logger or configure a new one.
- msnoise.api.get_engine(inifile=None)
Returns the a SQLAlchemy Engine
- msnoise.api.connect(inifile=None)
Establishes a connection to the database and returns a Session object.
- Parameters:
inifile (string) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini
- Return type:
- Returns:
A
Session
object, needed for many of the other API methods.
- msnoise.api.create_database_inifile(tech, hostname, database, username, password, prefix='')
Creates the db.ini file based on supplied parameters.
- Parameters:
tech (int) – The database technology used: 1=sqlite 2=mysql
hostname (string) – The hostname of the server (if tech=2) or the name of the sqlite file if tech=1)
database (string) – The database name
username (string) – The user name
prefix (string) – The prefix to use for all tables
password (string) – The password of user
- Returns:
None
- msnoise.api.read_db_inifile(inifile=None)
Reads the parameters from the db.ini file.
- Parameters:
inifile (string) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini
- Return type:
tuple
- Returns:
tech, hostname, database, username, password
- msnoise.api.get_config(session, name=None, isbool=False, plugin=None)
Get the value of one or all config bits from the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
name (str) – The name of the config bit to get. If omitted, a dictionnary with all config items will be returned
isbool (bool) – if True, returns True/False for config name. Defaults to False
plugin (str) – if provided, gives the name of the Plugin config to use. E.g. if “Amazing” is provided, MSNoise will try to load the “AmazingConfig” entry point. See Extending MSNoise with Plugins for details.
- Return type:
- Returns:
the value for name or a dict of all config values
- msnoise.api.update_config(session, name, value, plugin=None)
Update one config bit in the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
name (str) – The name of the config bit to set.
value (str) – The value of parameter name. Can also be NULL if you don’t want to use this particular parameter.
plugin (str) – if provided, gives the name of the Plugin config to use. E.g. if “Amazing” is provided, MSNoise will try to load the “AmazingConfig” entry point. See Extending MSNoise with Plugins for details.
- msnoise.api.get_params(session)
Get config parameters from the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Returns:
a Param class containing the parameters
- msnoise.api.get_filters(session, all=False, ref=None)
Get Filters from the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
all (bool) – Returns all filters from the database if True, or only filters where used = 1 if False (default)
- Return type:
list of
Filter
- Returns:
a list of Filter
- msnoise.api.update_filter(session, ref, low, mwcs_low, high, mwcs_high, mwcs_wlen, mwcs_step, used)
Updates or Insert a new Filter in the database.
See also
msnoise.msnoise_table_def.declare_tables.Filter
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
ref (int) – The id of the Filter in the database
low (float) – The lower frequency bound of the Whiten function (in Hz)
high (float) – The upper frequency bound of the Whiten function (in Hz)
mwcs_wlen (float) – Window length (in seconds) to perform MWCS
mwcs_step (float) – Step (in seconds) of the windowing procedure in MWCS
used (bool) – Is the filter activated for the processing
- msnoise.api.get_networks(session, all=False)
Get Networks from the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
all (bool) – Returns all networks from the database if True, or only networks at least one station has used = 1 if False (default)
- Return type:
list of str
- Returns:
a list of network codes
- msnoise.api.get_stations(session, all=False, net=None, format='raw')
Get Stations from the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
all (bool) – Returns all stations from the database if True, or only stations where used = 1 if False (default)
net (str) – if set, limits the stations returned to this network
- Return type:
list of
msnoise.msnoise_table_def.declare_tables.Station
- Returns:
list of
Station
- msnoise.api.get_station(session, net, sta)
Get one Station from the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
net (str) – the network code
sta (str) – the station code
- Return type:
msnoise.msnoise_table_def.declare_tables.Station
- Returns:
a
Station
Object
- msnoise.api.update_station(session, net, sta, X, Y, altitude, coordinates='UTM', instrument='N/A', used=1)
Updates or Insert a new Station in the database.
See also
msnoise.msnoise_table_def.declare_tables.Station
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
net (str) – The network code of the Station
sta (str) – The station code
X (float) – The X coordinate of the station
Y (float) – The Y coordinate of the station
altitude (float) – The altitude of the station
coordinates (str) – The coordinates system. “DEG” is WGS84 latitude/ longitude in degrees. “UTM” is expressed in meters.
instrument (str) – The instrument code, useful with PAZ correction
used (bool) – Whether this station must be used in the computations.
- msnoise.api.get_station_pairs(session, used=None, net=None)
Returns an iterator over all possible station pairs. If auto-correlation is configured in the database, returns N*N pairs, otherwise returns N*(N-1)/2 pairs.
- Parameters:
- Return type:
iterable
- Returns:
An iterable of
Station
object pairs
- msnoise.api.check_stations_uniqueness(session, station)
- Parameters:
session –
station –
- Returns:
- msnoise.api.get_interstation_distance(station1, station2, coordinates='DEG')
Returns the distance in km between station1 and station2.
Warning
Currently the stations coordinates system have to be the same!
- msnoise.api.update_data_availability(session, net, sta, loc, chan, path, file, starttime, endtime, data_duration, gaps_duration, samplerate)
Updates a DataAvailability object in the database
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
net (str) – The network code of the Station
sta (str) – The station code
chan (str) – The component (channel)
path (str) – The full path to the folder containing the file
file (str) – The name of the file
starttime (datetime.datetime) – Start time of the file
endtime (datetime.datetime) – End time of the file
data_duration (float) – Cumulative duration of available data in the file
gaps_duration (float) – Cumulative duration of gaps in the file
samplerate (float) – Sample rate of the data in the file (in Hz)
- msnoise.api.get_new_files(session)
Returns the files marked “N”ew or “M”odified in the database
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Return type:
list
- Returns:
list of
DataAvailability
- msnoise.api.get_data_availability(session, net=None, sta=None, loc=None, chan=None, starttime=None, endtime=None)
Returns the
DataAvailability
objects for specific net, sta, starttime or endtime- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
net (str) – Network code
sta (str) – Station code
starttime (datetime.datetime, datetime.date) – Start time of the search
endtime (datetime.datetime, datetime.date) – End time of the search
- Return type:
list
- Returns:
list of
DataAvailability
- msnoise.api.mark_data_availability(session, net, sta, flag)
Updates the flag of all
DataAvailability
objects matching net.sta in the database
- msnoise.api.count_data_availability_flags(session)
Count the number of
DataAvailability
, grouped by flag- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Return type:
list
- Returns:
list of [count, flag] pairs
- msnoise.api.update_job(session, day, pair, jobtype, flag, commit=True, returnjob=True, ref=None)
Updates or Inserts a new
Job
in the database.- Parameters:
day (str) – The day in YYYY-MM-DD format
pair (str) – the name of the pair (EXAMPLE?)
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.
commit (bool) – Whether to directly commit (True, default) or not (False)
returnjob (bool) – Return the modified/inserted Job (True, default) or not (False)
- Return type:
Job
or None- Returns:
If returnjob is True, returns the modified/inserted Job.
- msnoise.api.massive_insert_job(jobs)
Routine to use a low level function to insert much faster a list of
Job
. This method uses the Engine directly, no need to pass a Session object.- Parameters:
jobs (list) – a list of
Job
to insert.
- msnoise.api.massive_update_job(session, jobs, flag='D')
Routine to use a low level function to update much faster a list of
Job
. This method uses the Job.ref which is unique. :type session: Session :param session: the database connection object :type jobs: list or tuple :param jobs: a list ofJob
to update. :type flag: str :param flag: The destination flag.
- msnoise.api.is_next_job(session, flag='T', jobtype='CC')
Are there any
Job
in the database, with flag=`flag` and jobtype=`type`
- msnoise.api.get_next_job(session, flag='T', jobtype='CC', limit=99999)
Get the next
Job
in the database, with flag=`flag` and jobtype=`jobtype`. Jobs of the same type are grouped per day. This function also sets the flag of all selected Jobs to “I”n progress.- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.
- Return type:
list
- Returns:
list of
Job
- msnoise.api.get_dvv_jobs(session, flag='T', jobtype='DVV', limit=99999)
- msnoise.api.is_dtt_next_job(session, flag='T', jobtype='DTT', ref=False)
Are there any DTT
Job
in the database, with flag=`flag` and jobtype=`jobtype`. If ref is provided, checks if a DTT “REF” job is present.- Parameters:
- Return type:
- Returns:
True if at least one Job matches, False otherwise.
- msnoise.api.get_dtt_next_job(session, flag='T', jobtype='DTT')
Get the next DTT
Job
in the database, with flag=`flag` and jobtype=`jobtype`. Jobs are then grouped per station pair. This function also sets the flag of all selected Jobs to “I”n progress.- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.
- Return type:
tuple
- Returns:
(pairs, days, refs): List of station pair names - Days of the next DTT jobs - Job IDs (for later being able to update their flag).
- msnoise.api.reset_jobs(session, jobtype, alljobs=False, rule=None)
Sets the flag of all jobtype Jobs to “T”odo.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
alljobs (bool) – If True, resets all jobs. If False (default), only resets jobs “I”n progress.
- msnoise.api.reset_dtt_jobs(session, pair)
Sets the flag of all DTT Jobs of one pair to “T”odo.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
pair (str) – The pair to update
- msnoise.api.get_job_types(session, jobtype='CC')
Count the number of Jobs of a specific type, grouped by flag.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
- Return type:
list
- Returns:
list of [count, flag] pairs
- msnoise.api.get_jobs_by_lastmod(session, jobtype='CC', lastmod=datetime.datetime(2024, 2, 7, 14, 13, 32, 389030))
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
lastmod (datetime.datetime) – Jobs’ modification time
- Return type:
list
- Returns:
list of Job objects.
- msnoise.api.export_allcorr(session, ccfid, data)
- msnoise.api.export_allcorr2(session, ccfid, data)
- msnoise.api.add_corr(session, station1, station2, filterid, date, time, duration, components, CF, sampling_rate, day=False, ncorr=0, params=None)
Adds a CCF to the data archive on disk.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
station1 (str) – The name of station 1 (formatted NET.STA)
station2 (str) – The name of station 2 (formatted NET.STA)
filterid (int) – The ID (ref) of the filter
date (datetime.date or str) – The date of the CCF
time (datetime.time or str) – The time of the CCF
duration (float) – The total duration of the exported CCF
components (str) – The name of the components used (ZZ, ZR, …)
sampling_rate (float) – The sampling rate of the exported CCF
day (bool) – Whether this function is called to export a daily stack (True) or each CCF (when keep_all parameter is set to True in the configuration). Defaults to True.
ncorr (int) – Number of CCF that have been stacked for this CCF.
params (dict) – A dictionnary of MSNoise config parameters as returned by
get_params()
.
- msnoise.api.export_sac(db, filename, pair, components, filterid, corr, ncorr=0, sac_format=None, maxlag=None, cc_sampling_rate=None, params=None)
- msnoise.api.export_mseed(db, filename, pair, components, filterid, corr, ncorr=0, maxlag=None, cc_sampling_rate=None, params=None)
- msnoise.api.stack(data, stack_method='linear', pws_timegate=10.0, pws_power=2, goal_sampling_rate=20.0)
- Parameters:
data (
numpy.ndarray
) – the data to stack, each row being one CCFstack_method (str) – either
linear
: average of all CCF orpws
to compute the phase weigthed stack. Ifpws
is selected, the function expects thepws_timegate
andpws_power
.pws_timegate (float) – PWS time gate in seconds. Width of the smoothing window to convolve with the PWS spectrum.
pws_power (float) – Power of the PWS weights to be applied to the CCF stack.
goal_sampling_rate (float) – Sampling rate of the CCF array submitted
- Return type:
numpy.array
- Returns:
the stacked CCF.
- msnoise.api.get_extension(export_format)
- msnoise.api.get_ref(session, station1, station2, filterid, components, params=None)
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
station1 (str) – The name of station 1 (formatted NET.STA)
station2 (str) – The name of station 2 (formatted NET.STA)
filterid (int) – The ID (ref) of the filter
components (str) – The name of the components used (ZZ, ZR, …)
params (dict) – A dictionnary of MSNoise config parameters as returned by
get_params()
.
- Return type:
obspy.trace
- Returns:
A Trace object containing the ref
- msnoise.api.get_results(session, station1, station2, filterid, components, dates, mov_stack=1, format='stack', params=None)
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
station1 (str) – The name of station 1 (formatted NET.STA)
station2 (str) – The name of station 2 (formatted NET.STA)
filterid (int) – The ID (ref) of the filter
components (str) – The name of the components used (ZZ, ZR, …)
dates (list) – List of TODO datetime.datetime
mov_stack (int) – Moving window stack.
format (str) – Either
stack
: the data will be stacked according to the parameters passed withparams
ormatrix
: to get a 2D array of CCF.params (dict) – A dictionnary of MSNoise config parameters as returned by
get_params()
.
- Return type:
- Returns:
Either a 1D CCF (if format is
stack
or a 2D array (if format=matrix
).
- msnoise.api.get_mwcs(session, station1, station2, filterid, components, date, mov_stack=1)
TODO
- msnoise.api.get_results_all(session, station1, station2, filterid, components, dates)
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
station1 (str) – The name of station 1 (formatted NET.STA)
station2 (str) – The name of station 2 (formatted NET.STA)
filterid (int) – The ID (ref) of the filter
components (str) – The name of the components used (ZZ, ZR, …)
dates (list) – List of TODO datetime.datetime
- Return type:
- Returns:
All CCF results in a
pandas.DataFrame
, where the index is the time of the CCF and the columns are the times in the coda.
- msnoise.api.get_maxlag_samples(session)
Returns the length of the CC functions. Gets the maxlag and sampling rate from the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Return type:
- Returns:
the length of the CCF in samples
- msnoise.api.get_t_axis(session)
Returns the time axis (in seconds) of the CC functions. Gets the maxlag from the database and uses get_maxlag_samples function.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Return type:
numpy.array
- Returns:
the time axis in seconds
- msnoise.api.get_components_to_compute(session, plugin=None)
Returns the components configured in the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Return type:
list of str
- Returns:
a list of components to compute
- msnoise.api.get_components_to_compute_single_station(session, plugin=None)
Returns the components configured in the database.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Return type:
list of str
- Returns:
a list of components to compute
- msnoise.api.build_ref_datelist(session)
Creates a date array for the REF. The returned tuple contains a start and an end date, and a list of individual dates between the two.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Return type:
tuple
- Returns:
(start, end, datelist)
- msnoise.api.build_movstack_datelist(session)
Creates a date array for the analyse period. The returned tuple contains a start and an end date, and a list of individual dates between the two.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Return type:
tuple
- Returns:
(start, end, datelist)
- msnoise.api.updated_days_for_dates(session, date1, date2, pair, jobtype='CC', interval=datetime.timedelta(days=1), returndays=False)
Determines if any Job of jobtype=`jobtype` and for pair=`pair`, concerning a date between date1 and date2 has been modified in the last interval=`interval`.
- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
date1 (datetime.datetime) – Beginning of the period of interest
date2 (datetime.datetime) – End of the period of interest
pair (str) – Pair of interest
jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
interval (datetime.timedelta) – Interval of time before now to search for updated days
returndays (bool) – Whether to return a list of days (True) or not (False, default)
- Return type:
list or bool
- Returns:
List of days if returndays is True, only “True” if not. (not clear!)
- msnoise.api.azimuth(coordinates, x0, y0, x1, y1)
Returns the azimuth between two coordinate sets.
- msnoise.api.nextpow2(x)
Returns the next power of 2 of x.
- msnoise.api.check_and_phase_shift(trace, taper_length=20.0)
- msnoise.api.getGaps(stream, min_gap=None, max_gap=None)
- msnoise.api.make_same_length(st)
This function takes a stream of equal sampling rate and makes sure that all channels have the same length and the same gaps.
- msnoise.api.preload_instrument_responses(session, return_format='dataframe')
This function preloads all instrument responses from
response_format
and stores the seed ids, start and end dates, and paz for every channel in a DataFrame.Warning
This function only works for
response_format
being “inventory” or “dataless”.- Parameters:
session (
sqlalchemy.orm.session.Session
) – ASession
object, as obtained byconnect()
- Return type:
- Returns:
A table containing all channels with the time of operation and poles and zeros.
- msnoise.api.to_sds(stats, year, jday)
- msnoise.api.psd_read_results(net, sta, loc, chan, datelist, format='PPSD', use_cache=True)
- msnoise.api.psd_ppsd_to_dataframe(ppsd)
- msnoise.api.hdf_open_store_from_fn(fn, mode='a')
- msnoise.api.hdf_open_store(filename, location='PSD\\HDF', mode='a', format='table')
- msnoise.api.hdf_insert_or_update(store, key, new)
- msnoise.api.hdf_close_store(store)
- msnoise.api.xr_create_or_open(fn, taxis=[], name='CCF')
- msnoise.api.xr_insert_or_update(dataset, new)
- msnoise.api.xr_save_and_close(dataset, fn)
- msnoise.api.get_dvv(session, filterid, components, dates, mov_stack=1, aggregation='median', dttname='M')
- msnoise.api.xr_save_ccf(station1, station2, components, filterid, mov_stack, taxis, new, overwrite=False)
- msnoise.api.xr_get_ccf(station1, station2, components, filterid, mov_stack, taxis)
- msnoise.api.xr_save_ref(station1, station2, components, filterid, taxis, new, overwrite=False)
- msnoise.api.xr_get_ref(station1, station2, components, filterid, taxis)
- msnoise.api.xr_save_mwcs(station1, station2, components, filterid, mov_stack, taxis, dataframe)
- msnoise.api.xr_get_mwcs(station1, station2, components, filterid, mov_stack)
- msnoise.api.xr_save_dtt(station1, station2, components, filterid, mov_stack, dataframe)
- msnoise.api.xr_get_dtt(station1, station2, components, filterid, mov_stack)
- msnoise.api.xr_save_dvv(components, filterid, mov_stack, dataframe)
- msnoise.api.xr_get_dvv(components, filterid, mov_stack)
- msnoise.api.wavg(group, dttname, errname)
- msnoise.api.wstd(group, dttname, errname)
- msnoise.api.get_wavgwstd(data, dttname, errname)
- msnoise.api.trim(data, dttname, errname, limits=0.1)
- msnoise.api.compute_dvv(session, filterid, mov_stack, pairs=None, components=None, params=None, method=None, **kwargs)