MSNoise API

msnoise.api.get_logger(name, loglevel=None, with_pid=False)

Returns the current configured logger or configure a new one.

msnoise.api.get_engine(inifile=None)

Returns the a SQLAlchemy Engine

Parameters:inifile (str) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini
Return type:sqlalchemy.engine.Engine
Returns:An Engine Object
msnoise.api.connect(inifile=None)

Establishes a connection to the database and returns a Session object.

Parameters:inifile (string) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini
Return type:sqlalchemy.orm.session.Session
Returns:A Session object, needed for many of the other API methods.
msnoise.api.create_database_inifile(tech, hostname, database, username, password, prefix=”)

Creates the db.ini file based on supplied parameters.

Parameters:
  • tech (int) – The database technology used: 1=sqlite 2=mysql
  • hostname (string) – The hostname of the server (if tech=2) or the name of the sqlite file if tech=1)
  • database (string) – The database name
  • username (string) – The user name
  • prefix (string) – The prefix to use for all tables
  • password (string) – The password of user
Returns:

None

msnoise.api.read_db_inifile(inifile=None)

Reads the parameters from the db.ini file.

Parameters:inifile (string) – The path to the db.ini file to use. Defaults to os.cwd() + db.ini
Return type:tuple
Returns:tech, hostname, database, username, password
msnoise.api.get_config(session, name=None, isbool=False, plugin=None)

Get the value of one or all config bits from the database.

Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • name (str) – The name of the config bit to get. If omitted, a dictionnary with all config items will be returned
  • isbool (bool) – if True, returns True/False for config name. Defaults to False
  • plugin (str) – if provided, gives the name of the Plugin config to use. E.g. if “Amazing” is provided, MSNoise will try to load the “AmazingConfig” entry point. See Extending MSNoise with Plugins for details.
Return type:

str, bool or dict

Returns:

the value for name or a dict of all config values

msnoise.api.update_config(session, name, value, plugin=None)

Update one config bit in the database.

Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • name (str) – The name of the config bit to set.
  • value (str) – The value of parameter name. Can also be NULL if you don’t want to use this particular parameter.
  • plugin (str) – if provided, gives the name of the Plugin config to use. E.g. if “Amazing” is provided, MSNoise will try to load the “AmazingConfig” entry point. See Extending MSNoise with Plugins for details.
msnoise.api.get_params(session)

Get config parameters from the database.

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Returns:a Param class containing the parameters
msnoise.api.get_filters(session, all=False)

Get Filters from the database.

Parameters:
Return type:

list of Filter

Returns:

a list of Filter

msnoise.api.update_filter(session, ref, low, mwcs_low, high, mwcs_high, rms_threshold, mwcs_wlen, mwcs_step, used)

Updates or Insert a new Filter in the database.

See also

msnoise.msnoise_table_def.declare_tables.Filter

Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • ref (int) – The id of the Filter in the database
  • low (float) – The lower frequency bound of the Whiten function (in Hz)
  • high (float) – The upper frequency bound of the Whiten function (in Hz)
  • rms_threshold (float) – Not used anymore
  • mwcs_wlen (float) – Window length (in seconds) to perform MWCS
  • mwcs_step (float) – Step (in seconds) of the windowing procedure in MWCS
  • used (bool) – Is the filter activated for the processing
msnoise.api.get_networks(session, all=False)

Get Networks from the database.

Parameters:
Return type:

list of str

Returns:

a list of network codes

msnoise.api.get_stations(session, all=False, net=None)

Get Stations from the database.

Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • all (bool) – Returns all stations from the database if True, or only stations where used = 1 if False (default)
  • net (str) – if set, limits the stations returned to this network
Return type:

list of msnoise.msnoise_table_def.declare_tables.Station

Returns:

list of Station

msnoise.api.get_station(session, net, sta)

Get one Station from the database.

Parameters:
Return type:

msnoise.msnoise_table_def.declare_tables.Station

Returns:

a Station Object

msnoise.api.update_station(session, net, sta, X, Y, altitude, coordinates=’UTM’, instrument=’N/A’, used=1)

Updates or Insert a new Station in the database.

See also

msnoise.msnoise_table_def.declare_tables.Station

Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • net (str) – The network code of the Station
  • sta (str) – The station code
  • X (float) – The X coordinate of the station
  • Y (float) – The Y coordinate of the station
  • altitude (float) – The altitude of the station
  • coordinates (str) – The coordinates system. “DEG” is WGS84 latitude/ longitude in degrees. “UTM” is expressed in meters.
  • instrument (str) – The instrument code, useful with PAZ correction
  • used (bool) – Whether this station must be used in the computations.
msnoise.api.get_station_pairs(session, used=None, net=None)

Returns an iterator over all possible station pairs. If auto-correlation is configured in the database, returns N*N pairs, otherwise returns N*(N-1)/2 pairs.

Parameters:
Return type:

iterable

Returns:

An iterable of Station object pairs

msnoise.api.get_interstation_distance(station1, station2, coordinates=’DEG’)

Returns the distance in km between station1 and station2.

Warning

Currently the stations coordinates system have to be the same!

Parameters:
  • station1 (Station) – A Station object
  • station2 (Station) – A Station object
  • coordinates (str) – The coordinates system. “DEG” is WGS84 latitude/ longitude in degrees. “UTM” is expressed in meters.
Return type:

float

Returns:

The interstation distance in km

msnoise.api.update_data_availability(session, net, sta, comp, path, file, starttime, endtime, data_duration, gaps_duration, samplerate)

Updates a DataAvailability object in the database

Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • net (str) – The network code of the Station
  • sta (str) – The station code
  • comp (str) – The component (channel)
  • path (str) – The full path to the folder containing the file
  • file (str) – The name of the file
  • starttime (datetime.datetime) – Start time of the file
  • endtime (datetime.datetime) – End time of the file
  • data_duration (float) – Cumulative duration of available data in the file
  • gaps_duration (float) – Cumulative duration of gaps in the file
  • samplerate (float) – Sample rate of the data in the file (in Hz)
msnoise.api.get_new_files(session)

Returns the files marked “N”ew or “M”odified in the database

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type:list
Returns:list of DataAvailability
msnoise.api.get_data_availability(session, net=None, sta=None, comp=None, starttime=None, endtime=None)

Returns the DataAvailability objects for specific net, sta, starttime or endtime

Parameters:
Return type:

list

Returns:

list of DataAvailability

msnoise.api.mark_data_availability(session, net, sta, flag)

Updates the flag of all DataAvailability objects matching net.sta in the database

Parameters:
msnoise.api.count_data_availability_flags(session)

Count the number of DataAvailability, grouped by flag

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type:list
Returns:list of [count, flag] pairs
msnoise.api.update_job(session, day, pair, jobtype, flag, commit=True, returnjob=True, ref=None)

Updates or Inserts a new Job in the database.

Parameters:
  • day (str) – The day in YYYY-MM-DD format
  • pair (str) – the name of the pair (EXAMPLE?)
  • jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
  • flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.
  • commit (bool) – Whether to directly commit (True, default) or not (False)
  • returnjob (bool) – Return the modified/inserted Job (True, default) or not (False)
Return type:

Job or None

Returns:

If returnjob is True, returns the modified/inserted Job.

msnoise.api.massive_insert_job(jobs)

Routine to use a low level function to insert much faster a list of Job. This method uses the Engine directly, no need to pass a Session object.

Parameters:jobs (list) – a list of Job to insert.
msnoise.api.massive_update_job(session, jobs, flag=’D’)

Routine to use a low level function to update much faster a list of Job. This method uses the Job.ref which is unique.

Parameters:
  • jobs (list) – a list of Job to update.
  • flag (str) – The destination flag.
msnoise.api.is_next_job(session, flag=’T’, jobtype=’CC’)

Are there any Job in the database, with flag=`flag` and jobtype=`type`

Parameters:
Return type:

bool

Returns:

True if at least one Job matches, False otherwise.

msnoise.api.get_next_job(session, flag=’T’, jobtype=’CC’)

Get the next Job in the database, with flag=`flag` and jobtype=`jobtype`. Jobs of the same type are grouped per day. This function also sets the flag of all selected Jobs to “I”n progress.

Parameters:
Return type:

list

Returns:

list of Job

msnoise.api.is_dtt_next_job(session, flag=’T’, jobtype=’DTT’, ref=False)

Are there any DTT Job in the database, with flag=`flag` and jobtype=`jobtype`. If ref is provided, checks if a DTT “REF” job is present.

Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • jobtype (str) – CrossCorrelation (CC) or dt/t (DTT) Job?
  • flag (str) – Status of the Job: “T”odo, “I”n Progress, “D”one.
  • ref (bool) – Whether to check for a REF job (True) or not (False, default)
Return type:

bool

Returns:

True if at least one Job matches, False otherwise.

msnoise.api.get_dtt_next_job(session, flag=’T’, jobtype=’DTT’)

Get the next DTT Job in the database, with flag=`flag` and jobtype=`jobtype`. Jobs are then grouped per station pair. This function also sets the flag of all selected Jobs to “I”n progress.

Parameters:
Return type:

tuple

Returns:

(pairs, days, refs): List of station pair names - Days of the next DTT jobs - Job IDs (for later being able to update their flag).

msnoise.api.reset_jobs(session, jobtype, alljobs=False, rule=None)

Sets the flag of all jobtype Jobs to “T”odo.

Parameters:
msnoise.api.reset_dtt_jobs(session, pair)

Sets the flag of all DTT Jobs of one pair to “T”odo.

Parameters:
msnoise.api.get_job_types(session, jobtype=’CC’)

Count the number of Jobs of a specific type, grouped by flag.

Parameters:
Return type:

list

Returns:

list of [count, flag] pairs

msnoise.api.get_jobs_by_lastmod(session, jobtype=’CC’, lastmod=datetime.datetime(2019, 1, 10, 17, 5, 42, 611029))
Parameters:
Return type:

list

Returns:

list of Job objects.

msnoise.api.export_allcorr(session, ccfid, data)
msnoise.api.export_allcorr2(session, ccfid, data)
msnoise.api.add_corr(session, station1, station2, filterid, date, time, duration, components, CF, sampling_rate, day=False, ncorr=0, params=None)

Adds a CCF to the data archive on disk.

Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • station1 (str) – The name of station 1 (formatted NET.STA)
  • station2 (str) – The name of station 2 (formatted NET.STA)
  • filterid (int) – The ID (ref) of the filter
  • date (datetime.date or str) – The date of the CCF
  • time (datetime.time or str) – The time of the CCF
  • duration (float) – The total duration of the exported CCF
  • components (str) – The name of the components used (ZZ, ZR, …)
  • sampling_rate (float) – The sampling rate of the exported CCF
  • day (bool) – Whether this function is called to export a daily stack (True) or each CCF (when keep_all parameter is set to True in the configuration). Defaults to True.
  • ncorr (int) – Number of CCF that have been stacked for this CCF.
  • params (dict) – A dictionnary of MSNoise config parameters as returned by get_params().
msnoise.api.export_sac(db, filename, pair, components, filterid, corr, ncorr=0, sac_format=None, maxlag=None, cc_sampling_rate=None, params=None)
msnoise.api.export_mseed(db, filename, pair, components, filterid, corr, ncorr=0, maxlag=None, cc_sampling_rate=None, params=None)
msnoise.api.stack(data, stack_method=’linear’, pws_timegate=10.0, pws_power=2, goal_sampling_rate=20.0)
Parameters:
  • data (numpy.ndarray) – the data to stack, each row being one CCF
  • stack_method (str) – either linear: average of all CCF or pws to compute the phase weigthed stack. If pws is selected, the function expects the pws_timegate and pws_power.
  • pws_timegate (float) – PWS time gate in seconds. Width of the smoothing window to convolve with the PWS spectrum.
  • pws_power (float) – Power of the PWS weights to be applied to the CCF stack.
  • goal_sampling_rate (float) – Sampling rate of the CCF array submitted
Return type:

numpy.array

Returns:

the stacked CCF.

msnoise.api.get_results(session, station1, station2, filterid, components, dates, mov_stack=1, format=’stack’, params=None)
Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • station1 (str) – The name of station 1 (formatted NET_STA)
  • station2 (str) – The name of station 2 (formatted NET_STA)
  • filterid (int) – The ID (ref) of the filter
  • components (str) – The name of the components used (ZZ, ZR, …)
  • dates (list) – List of TODO datetime.datetime
  • mov_stack (int) – Moving window stack.
  • format (str) – Either stack: the data will be stacked according to the parameters passed with params or matrix: to get a 2D array of CCF.
  • params (dict) – A dictionnary of MSNoise config parameters as returned by get_params().
Return type:

numpy.ndarray

Returns:

Either a 1D CCF (if format is stack or a 2D array (if format= matrix).

msnoise.api.get_results_all(session, station1, station2, filterid, components, dates)
Parameters:
  • session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
  • station1 (str) – The name of station 1 (formatted NET_STA)
  • station2 (str) – The name of station 2 (formatted NET_STA)
  • filterid (int) – The ID (ref) of the filter
  • components (str) – The name of the components used (ZZ, ZR, …)
  • dates (list) – List of TODO datetime.datetime
Return type:

pandas.DataFrame

Returns:

All CCF results in a pandas.DataFrame, where the index is the time of the CCF and the columns are the times in the coda.

msnoise.api.get_maxlag_samples(session)

Returns the length of the CC functions. Gets the maxlag and sampling rate from the database.

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type:int
Returns:the length of the CCF in samples
msnoise.api.get_t_axis(session)

Returns the time axis (in seconds) of the CC functions. Gets the maxlag from the database and uses get_maxlag_samples function.

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type:numpy.array
Returns:the time axis in seconds
msnoise.api.get_components_to_compute(session, plugin=None)

Returns the components configured in the database.

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type:list of str
Returns:a list of components to compute
msnoise.api.get_components_to_compute_single_station(session, plugin=None)

Returns the components configured in the database.

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type:list of str
Returns:a list of components to compute
msnoise.api.build_ref_datelist(session)

Creates a date array for the REF. The returned tuple contains a start and an end date, and a list of individual dates between the two.

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type:tuple
Returns:(start, end, datelist)
msnoise.api.build_movstack_datelist(session)

Creates a date array for the analyse period. The returned tuple contains a start and an end date, and a list of individual dates between the two.

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type:tuple
Returns:(start, end, datelist)
msnoise.api.updated_days_for_dates(session, date1, date2, pair, jobtype=’CC’, interval=datetime.timedelta(1), returndays=False)

Determines if any Job of jobtype=`jobtype` and for pair=`pair`, concerning a date between date1 and date2 has been modified in the last interval=`interval`.

Parameters:
Return type:

list or bool

Returns:

List of days if returndays is True, only “True” if not. (not clear!)

msnoise.api.azimuth(coordinates, x0, y0, x1, y1)

Returns the azimuth between two coordinate sets.

Parameters:
  • coordinates (str) – {‘DEG’, ‘UTM’, ‘MIX’}
  • x0 (float) – X coordinate of station 1
  • y0 (float) – Y coordinate of station 1
  • x1 (float) – X coordinate of station 2
  • y1 (float) – Y coordinate of station 2
Return type:

float

Returns:

The azimuth in degrees

msnoise.api.nextpow2(x)

Returns the next power of 2 of x.

Parameters:x (int) – any value
Return type:int
Returns:the next power of 2 of x
msnoise.api.check_and_phase_shift(trace)
msnoise.api.getGaps(stream, min_gap=None, max_gap=None)
msnoise.api.make_same_length(st)

This function takes a stream of equal sampling rate and makes sure that all channels have the same length and the same gaps.

msnoise.api.clean_scipy_cache()

This functions wraps all destroy scipy cache at once. It is a workaround to the memory leak induced by the “caching” functions in scipy fft.

msnoise.api.preload_instrument_responses(session)

This function preloads all instrument responses from response_format and stores the seed ids, start and end dates, and paz for every channel in a DataFrame.

Warning

This function only works for response_format being “inventory” or “dataless”.

Parameters:session (sqlalchemy.orm.session.Session) – A Session object, as obtained by connect()
Return type:pandas.DataFrame
Returns:A table containing all channels with the time of operation and poles and zeros.