ccatkidlib.analysis.core package
Submodules
ccatkidlib.analysis.core.data module
This library defines the Data base class which represents a raw RFSoC data product produced using the ccatkidlib data acquisition library.
The Data class implements attributes/methods that are universal between all types of RFSoC data products
and should be sub-classed to handle functionality (e.g., data loading) that differs between different data products (e.g., frequency sweep vs. time-ordered).
- class ccatkidlib.analysis.core.data.Data(com_to: str, cfg_path: str = '/home/docs/checkouts/readthedocs.org/user_builds/ccatkidlib/envs/latest/lib/python3.12/site-packages/ccatkidlib/analysis/analysis_config.yaml', data_type: Literal['vna', 'targ', 'timestream'] | None = None, timestamp: int | str | None = None, data_path: str | Path | list[str] | list[Path] | None = None, analysis_cfg: dict | None = None, viz_cfg: dict | None = None, **kwargs)
Bases:
objectClass representing a raw RFSoC data product produced using the ccatkidlib data acquisition library
- tones
List of tones that are loaded. None for data products with data not split into individual tones
- Type:
list[int] | None
- padding
Number of digits of the tone suffix in
dataDataFrame column names. For example, the tone suffix for tone 7 would be 007 for apaddingof three- Type:
int
- data_path
Path to data file
- Type:
Path
- timestamp
Timestamp of data file
- Type:
int
- analysis_cfg
Loaded analysis config
- Type:
dict
- viz_cfg
Loaded visualization config
- Type:
dict
Initialize Data object by finding RFSoC data file(s) and associated configuration files. Neither the RFSoC data file(s) nor the configuration files are loaded during initialization; it is only verified that the files exist.
Important
Either the full path(s) of the RFSoC data file(s) must be provided via the
data_pathkeyword argument or both thetimestampanddata_typemust be provided. All other segments of the path are optional.- Parameters:
analysis_cfg – Path to analysis configuration file
data_type – Type of RFSoC measurement
timestamp – Timestamp of RFSoC data file
data_path – Full path to RFSoC data file
**kwargs** – Key word arguments for finding RFSoC data file. See below:
root_data_dir (str) – Root directory of ccatkidlib file tree with RFSoC measurements. Defaults to that specified in the analysis configuration file
data_dir (str) – Name of directory in ccatkidlib file tree containing data file. Will typically correspond to the name of the measurement
date (str) – Date measurement was taken
sess_id (str) – ccatkidlib session ID of measurement
- Raises:
ValueError – If multiple data files are specified and they have different file extensions or timestamps
FileNotFoundError – If any data files cannot be found
- I(prefix: str | list[str] = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None) DataFrame
Get the in-phase \(I\) data
- Parameters:
prefix – Prefix(es) added to base column name I
include – List of tones to include
exclude – List of tones to exclude
- Returns:
Polars DataFrame with \(I\) data
- IQ_rotate(prefix: str | list[str] = '', angle: float | list[float] | list[list[float]] = 0, name: str = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) DataFrame
Rotate the in-phase \(I\) and quadrature \(Q\) data by
angle\(\theta\) around the origin. \(S_{21,\ rot} = I_{rot} + iQ_{rot} = (I + iQ)\cdot e^{i\theta}\)- Parameters:
prefix – Prefix(es) of the I and Q data columns to rotate
angle – Angle by which to rotate data. Can specify a list of angles for inidividual tones
name – Name of rotation operation. Will be prefixed to I and Q data column names
include – List of tones to include
exclude – List of tones to exclude
recalc – Whether to re-calculate if rotated I and Q data columns already exist
- Returns:
Polars DataFrame with rotated in-phase \(I\) and quadrature \(Q\) data
- IQ_scale(prefix: str | list[str] = '', scale: float | list[float] | list[list[float]] = 1, name: str = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) DataFrame
Divide the in-phase \(I\) and quadrature \(Q\) data by
scale- Parameters:
prefix – Prefix(es) of the I and Q data columns to scale
scale – Factor to scale by. Can specify list of scale factors for individual tones
name – Name of scaling operation. Will be prefixed to I and Q data column names
include – List of tones to include
exclude – List of tones to exclude
recalc – Whether to re-calculate if scaled I and Q data columns already exist
- Returns:
Polars DataFrame with scaled in-phase \(I\) and quadrature \(Q\) data
- IQ_shift(prefix: str | list[str] = '', shift_I: float | list[float] | list[list[float]] = 0, shift_Q: float | list[float] | list[list[float]] = 0, name: str = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) DataFrame
Translate the in-phase \(I\) data by
shift_Iand the quadrature \(Q\) data byshift_Q- Parameters:
prefix – Prefix(es) of the I and Q data columns to translate
shift_I – Value to translate \(I\) data by. Can specify list of shifts for individual tones
shift_Q – Value to translate \(Q\) data by. Can specify list of shifts for individual tones
name – Name of translation operation. Will be prefixed to I and Q data column names
include – List of tones to include
exclude – List of tones to exclude
recalc – Whether to re-calculate if translated I and Q data columns already exist
- Returns:
Polars DataFrame with translated in-phase \(I\) and quadrature \(Q\) data
- IQ_trim(prefix: str | list[str] = '', lower_index: int | list[int] | list[list[int]] = 0, upper_index: int | list[int] | list[list[int]] = -1, name: str = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) DataFrame
Trim the in-phase \(I\) and quadrature \(Q\) data to only include rows (samples) between
lower_indexandupper_index(exclusive). The trimmed data values are replaced with None.Note
Standard negative indexing is supported.
- Parameters:
prefix – Prefix(es) of the I and Q data columns to trim
lower_index – Lower index of trimming window (exclusive). Can specify list of indicies for individual tones
upper_index – Upper index of trimming window (exclusive). Can specify list of indicies for individual tones
name – Name of trimming operation. Will be prefixed to I and Q data column names
include – List of tones to include
exclude – List of tones to exclude
recalc – Whether to re-calculate if trimmed I and Q data columns already exist
- Returns:
Polars DataFrame with trimmed in-phase \(I\) and quadrature \(Q\) data
- Q(prefix: str | list[str] = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None) DataFrame
Get the quadrature \(Q\) data
- Parameters:
prefix – Prefix(es) added to base column name Q
include – List of tones to include
exclude – List of tones to exclude
- Returns:
Polars DataFrame with \(Q\) data
- property comb: DataFrame
Polars DataFrame with tone frequencies, powers, and phases of comb that was used to take data
- property data: DataFrame
Polars DataFrame with data. Load data if it is not already loaded.
- diff(col_name: str, prefix: str | list[str] = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) DataFrame
Calculate the difference between subsequent elements in the specified data column of the
dataPolars DataFrame- Parameters:
col_name – Data column name without prefix or tone suffix (e.g., mag, phase, etc.)
prefix – Prefix(es) of data column
include – List of tones to include
exclude – List of tones to exclude
recalc – Whether to re-calculate if difference data column already exists
- Returns:
Polars DataFrame with difference data
- property drone_cfg: dict
‘drone’ configuration file. Stores parameters that may differ between RFSoC drones
- property ext_cfg: dict
‘External’ configuration file. Stores parameters that may change between measurements and are not directly related to RFSoC drones
- property fig_dir: str
Directory where figures should be saved. Create if it does not already exist.
- get_data(col_name: str | list[str] = '.*', include: int | list[int] | None = None, exclude: int | list[int] | None = None, strict: bool = False) DataFrame
Get the specified data columns from the
dataPolars DataFrame- Parameters:
col_name – List of data column names to get without tone suffix (e.g., ‘mag’, ‘phase’, etc.)
include – List of tones to include
exclude – List of tones to exclude
strict – Whether the data column name(s) must match
col_nameexactly. If False, data column names with prefixes (not suffixes!) tocol_namewill also be included
- Returns:
Polars DataFrame with specified data columns
- get_prefix(col_name: str, data_name: str) str
Extract the prefix from the provided column name
col_name. Assumes that the column name is of the format {prefix}_{data_name}_{tone}- Parameters:
col_name – Full name of the column
data_name – Data type that column corresponds to (e.g., ‘phase’, ‘f’, etc.)
- Returns:
Prefix of the column name
- property io_cfg: dict
IO configuration file. Stores parameters related to networking, file, and logging IO operations
- join(other: Data, in_place: bool = False) Data
Join two
Dataobjects into a single object- Parameters:
other –
Dataobject to join within_place – Whether to perform join in-place (will modify this
Dataobject) or to create newDataobject
- Returns:
A reference to this
Dataobject ifin_placeis True, otherwise the newly createdDataobject
- mag(prefix: str | list[str] = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None, dB: bool = False, recalc: bool = False) DataFrame
Calculate the magnitude \(|S_{21}| = \sqrt{I^2 + Q^2}\) using the in-phase \(I\) and quadrature \(Q\) data
- Parameters:
prefix – Prefix(es) added to base column name mag
include – List of tones to include
exclude – List of tones to exclude
dB – Whether to calculate magnitude in decible units. \(|S_{21}|_{dB} = 20\cdot\log_{10}(|S_{21}|)\)
recalc – Whether to re-calculate if magnitude column already exists
- Returns:
Polars DataFrame with magnitude data
- property num_tones: int
Total number of tones that were used to take data
- phase(prefix: str | list[str] = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) DataFrame
Calculate the phase \(\arctan{(Q/I)}\) using the in-phase \(I\) and quadrature \(Q\) data
- Parameters:
prefix – Prefix(es) added to base column name phase
include – List of tones to include
exclude – List of tones to exclude
recalc – Whether to re-calculate if phase column already exists
- Returns:
Polars DataFrame with phase data
- property pickle_dir: str
Directory where pickle files should be saved. Create if it does not already exist.
- savgol(col_name: str, prefix: str | list[str] = '', window: int | list[int] | list[list[int]] = 3, k: int | list[int] | list[list[int]] = 1, deriv: int | list[int] | list[list[int]] = 0, include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, max_workers: int = 1, ex: ProcessPoolExecutor | None = None) pl.DataFrame
Apply Savitzky–Golay filter to specified data column
See also
The data is filtered using SciPy’s savgol_filter function
- Parameters:
col_name – Data column name without prefix or tone suffix (e.g., mag, phase, etc.)
prefix – Prefix(es) of data column
window – The length of the filter window. Can specify a list of windows for individual tones
k – Order of polynomial used to fit the data. Can specify a list of orders for individual tones
deriv – The order of derivative to compute. Can specify a list of orders for inidividual tones
include – List of tones to include
exclude – List of tones to exclude
recalc – Whether to re-calculate if Savitzky–Golay filtered data column already exists
max_workers – Max number of CPU cores to use for multiprocessing. Only used if no
exprovided.ex – concurrent.futures ProcessPoolExecutor to use for multiprocessing
- Returns:
Polars DataFrame with Savitzky–Golay filtered data
- transform(funcs: list[Callable[[Schema, Any], Expr | list[Expr]]], *funcs_args, include: int | list[int] | None = None, exclude: int | list[int] | None = None, col_name: str | list[str] | list[list[str]] = [], recalc: bool = False) DataFrame
Apply
funcstransformations to thecol_namedata columns in thedataPolars DataFrameNote
This method is mostly intended for internal use but can be used to run multiple types of transformations in a single Polars
.with_columnscall which may improve efficiency.Examples
To calculate the phase and magnitude for tones 1, 5, and 7, one would run:
>>> data.transform([Data._calc_phase, Data._calc_mag], include=[1,5,7], col_name=[['I', 'Q'], ['I', 'Q']])
- Parameters:
funcs – List of transformation functions to apply to data.
func_args – Positional arguments to pass to transformation functions. Must be a list of the same length as
funcswith each element being the positional arguments to pass to the corresponding transformation function.include – List of tones to include
exclude – List of tones to exclude
col_name – Data columns to apply transformation to
recalc – Whether to re-calculate if transformed data column already exists
- Returns:
Full
dataPolars DataFrame including transformed data columns
- unpivot(df: DataFrame, data_name: str, index_cols: list[str] = ['sample']) DataFrame
Unpivot (wide to long) data columns in the
dfPolars DataFrame. The DataFrame should only contain one type of data (e.g., ‘phase’) but can contain data for any number of tones. Any additional columns should be specified asindex_cols.See also
Documentation of Polars unpivot operation
- Parameters:
df – Polars DataFrame with data columns to unpivot
data_name – Type of data (e.g., ‘phase’, ‘f’, etc.)
index_cols – Additional columns in
dfthat should not be unpivoted
- Returns:
Unpivoted Polars DataFrame. The new DataFrame has a ‘det’ column specifiying which tone each row corresponds to
ccatkidlib.analysis.core.detector module
Module for analyzing kinetic inductance detector (KID) composite data
- Authors:
Darshan Patel <dp649@cornell.edu>
- class ccatkidlib.analysis.core.detector.Detector(com_to: str, cfg_path: str = '/home/docs/checkouts/readthedocs.org/user_builds/ccatkidlib/envs/latest/lib/python3.12/site-packages/ccatkidlib/analysis/analysis_config.yaml', dets: int | list[int] = -1, noise_tones: int | list[int] | None = None, cable_delay: float | None = None, stream: Timestream | None = None, stream_path: str | PosixPath | list[str] | list[PosixPath] | None = None, stream_timestamp: int | str | None = None, targ: Target | None = None, targ_path: str | PosixPath | None = None, targ_timestamp: int | str | None = None, vna: VNA | None = None, vna_path: str | PosixPath | None = None, vna_timestamp: int | str | None = None, analysis_cfg: dict | None = None, viz_cfg: dict | None = None, **kwargs)
Bases:
objectClass representing kinetic inductance detectors (KIDs). Used for KID analyses requiring fitting and/or multiple types of data files (e.g., timestream and target sweep data).
- bid
RFSoC board that took the detector data
- Type:
str
- drid
RFSoC drone that took the detector data
- Type:
str
- tones
List of detectors
- Type:
list[int]
- stream
Timestreamobject of detector timestream- Type:
Timestream | None
- analysis_cfg
Config file with parameters used for data analysis
- Type:
dict
- viz_cfg
Config file with paramateres used for data visualization
- Type:
dict
- cable_delay
Cable delay of the network (in nanoseconds)
- Type:
float | None
- properties
Polars dataframe with detector properties
- Type:
pl.DataFrame
Constructor for Detector. Creates ccatkidlib data objects (
VNA,Target,Timsetream)Note
A Detector object can be initialized without a
Timestreamobject but must always have aTargetobject.If only the information to load a timestream is provided, an attempt will be made to find the target sweep file corresponding to the timestream
- Parameters:
com_to (str) – Drone that took the data. In form ‘<board>.<drone>’
analysis_cfg (str, optional) – Path to analysis configuration file. Defaults to analysis configuration file in ccatkidlib/ccatkidlib/analysis
dets (int | list[int], optional) – Which detectors to load; -1 to load all detectors. Defaults to -1
noise_tones (int | list[int] | None, optional) – Indicies of noise tones (tones not placed on detectors). Defaults to None
cable_delay (float | None, optional) – Cable delay of full network. Defaults to None
stream (Timestream | None, optional) – Detector
Timestreamobject. Defaults to Nonestream_path (str | pathlib.PosixPath | list[str] | list[pathlib.PosixPath] | None, optional) – Data path to detector timestream files. Defaults to None
stream_timestamp (int | str | None, optional) – Timestamp of detector timestream files. Defaults to None
targ (Target | None, optional) – Detector
Targetobject. Defaults to Nonetarg_path (str | pathlib.PosixPath | None, optional) – Data path to detector target sweep file. Defaults to None
targ_timestamp (int | str | None, optional) – Timestamp of detector target sweep file. Defaults to None
vna (VNA | None, optional) – Detector
VNAobject. Defaults to Nonevna_path (str | pathlib.PosixPath | None, optional) – Data path to detector VNA sweep file. Defaults to None
vna_timestamp (int | str | None, optional) – Timestamp of detector VNA sweep file. Defaults to None
**kwargs – Key word arguments for finding data files. See below:
root_data_dir (str, optional) – Root directory where data is stored. Defaults to that specified in analysis config
data_dir (str, optional) – Directory where data is stored
date (str, optional) – Date data was taken
sess_id (str, optional) – ccatkidlib session ID of data
- IQ_circle_center()
Remove cable delay from IQ circle and center at the origin
- IQ_circle_fit(prefix: str | list[str] = 'unwind_rotate', bounds=None, loss: str = 'soft_l1', f_scale: float = 1, method: str = 'trf', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, max_workers=1, ex=None) DataFrame
Fit the target sweep circle in the IQ plane
- IQ_circle_real(prefix: str | list[str] = 'unwind_rotate', data: str = 'both', loc: str = 'origin', circle_fit_col='circle_fit_unwind_rotate', use_fit=True, include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) list[DataFrame]
Rotate and center the target sweep circle in the IQ plane onto the real axis
- IQ_circle_rotate(prefix: str | list[str] = 'origin_shift_origin_rotate_unwind_rotate', data: str = 'both', rotation: str = 'mismatch', mean_points: int = 10, include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, **kwargs) list[DataFrame]
Rotate the target sweep circle in the IQ plane around its center
- IQ_max_dist(trim_window: int = 2, trim_savgol_window: int = 9, diff_savgol_window: int = 21, trim_savgol_k: int = 1, diff_savgol_k: int = 1, include=None, exclude=None, recalc=False, max_workers=1, ex=None)
Get the frequency corresponding to the max geometric distance between adjacent points in IQ space
- IQ_noise(prefix: str | list[str] = 'mismatch_rotate_origin_shift_origin_rotate', use_noise_tones: bool = True, include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) DataFrame
Transform timestream data to isolate readout noise. Will shift nearest frequency noise tone onto detector tone if
use_noise_tones, otherwise will rotate detector tone by ninety degrees around its center- Parameters:
prefix (str | list[str])
use_noise_tones (bool) – Whether to use noise tones. Defaults to True
- IQ_norm(prefix: str | list[str] = '', data: str = 'both', norm_col: str = 'cable_complex_fit', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) list[DataFrame]
Divide out cable baseline from target sweep and/or timestream I & Q data
- IQ_trim(prefix: str | list[str] = '', window: float | list[float] = 1.5, use_fit: bool = False, f_0_col: str = 'complex_fit_f_0', Q_col: str = 'complex_fit_Q', mean_points: int = 10, mag_prefix: str = '', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) DataFrame
Trim off-resonance target sweep I & Q data
- Parameters:
prefix
window (float | list[float], optional) – How many linewidths of data to include around the magnitude minimum. Defaults to 1.5
use_fit (bool) – Whether to use resonant frequency and total quality factor from a fit to determine linewidth. Defaults to False
f_0_col (str) – Name of column in
propertiesDataFrame with resonant frequencies to use ifuse_fitis True.Q_col (str) – Name of column in
propertiesDataFrame with total quality factors to use ifuse_fitis True.mean_points (int) – Number of points to average to determine max magnitude of resonator profile. Used if
use_fitis False. Defaults to 10
- Returns:
Polars DataFrame with trimmed I & Q data
- Return type:
return (pl.DataFrame)
- IQ_unwind(prefix: str | list[str] = '', data: str = 'both', delay_col: str = 'network_cable_delay', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) list[DataFrame]
Remove cable delay from target sweep and/or timestream I & Q data
- Parameters:
prefix (str | list[str]) – Prefix of I & Q data to remove cable delay from
data (str, optional) – Which type of data to remove cable delay from. Options are ‘targ’ or ‘timestream’. Defaults to ‘targ’.
delay_col (str) – Column in
propertiesDataFrame with detector cable delaysinclude (int | list[int] | None, optional) – Detector(s) for which to perform calculation
exclude (int | list[int] | None, optional) – Detector(s) for which not to perform calculation
recalc (bool) – Whether to recalculate if data is already in
dataDataFrame. Defaults to False
- Returns:
Polars DataFrame with unwound I & Q data
- Return type:
pl.Dataframe
- property cable_delay
- static calc_IQ_circle_fit(schema, *args, tones: list[int], padding: int = 4, recalc: bool = False, col_name=['I', 'Q', 'circle_fit'])
- static calc_complex_fit(schema, *args, tones: list[int], padding: int = 4, recalc: bool = False, col_name=['f', 'I', 'Q', 'complex_fit'])
Fit using resonator_model_v3
- static calc_frac_f(schema, *args, tones: list[int], padding: int = 4, recalc: bool = False, col_name=['f', 'frac'])
- static calc_noise_shift(schema, *args, tones: list[int], padding: int = 4, recalc: bool = False, col_name=['I', 'Q', 'noise'])
- static calc_phase_fit(schema, *args, tones: list[int], padding: int = 4, recalc: bool = False, col_name=['f', 'I', 'Q', 'phase', 'phase_fit'])
- static calc_phase_spline(schema, *args, tones: list[int], padding: int = 4, recalc: bool = False, col_name=['f', 'phase', 'to_f', 'to_phase'])
- static calc_phase_to_f(schema, *args, tones: list[int], padding: int = 4, recalc: bool = False, col_name=['phase', 'f'])
- complex_fit(nonlinear: bool = False, asymm: bool = False, fix_cable: bool = False, fix_thetaQ: bool = False, include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, max_workers: int = 1, ex=None) DataFrame
Fit target sweep using complex forward transmission data (z = I + iQ)
- Parameters:
nonlinear (bool, optional) – Whether to perform a nonlinear fit. Defaults to False
asymm (bool, optional) – Whether to perform a asymmetric fit. Defaults to False
fix_cable (bool, optional) – Whether to vary cable parameters for fit. Defaults to False: parameters are varied
fix_thetaQ (bool, optional) – Whether to vary impedance mismatch angle and coupling quality factor for fit. Defaults to False: parameters are varied
- Returns:
Polars DataFrame with fit I and Q data
- Return type:
return (pl.DataFrame)
- property fig_dir: str
Directory where figures should be saved. Create if it does not already exist.
- property fit_dir
- frac_f(prefix: str | list[str] = 'mismatch_rotate_origin_shift_origin_rotate_unwind_rotate', f_0: str | list[float] = 'tone_freqs', name='', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, **kwargs) DataFrame
Convert timestream frequency data to fractional frequency shift
- get_properties(col_name: str | list[str] = '.*', include: int | list[int] | None = None, exclude: int | list[int] | None = None, strict: bool = False)
Get the specified data columns and rows from the
propertiesPolars DataFrame- Parameters:
col_name (str | list[str], optional) – Defaults to all columns
include (int | list[int] | None, optional) – Defaults to None
exclude (int | list[int] | None, optional) – Defaults to None
strict (bool, optional) – Defaults to False
- is_bifurcated(bifurcation_threshold=60, qifurcation_threshold=50, trim_window: int = 2, trim_savgol_window: int = 9, trim_savgol_k: int = 1, include=None, exclude=None, recalc=False, max_workers=1, ex=None)
Determine if a detector is bifurcated or has a high quasiparticle nonlinearity using the angle of the maximally seperated points in IQ space
- join(other, in_place=False)
- mag_min(include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False) list[DataFrame]
- phase_fit(prefix: str | list[str] = 'mismatch_rotate_origin_shift_origin_rotate_unwind_rotate', circle_fit_col: str = 'circle_fit_unwind_rotate', nonlinear: bool = False, method: str = 'least_squares', params: Parameters = None, window: float = 1, include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, max_workers: int = 1, ex=None) DataFrame
Fit target sweep using phase data (arctan(Q/I))
- phase_spline(prefix: str | list[str] = 'mismatch_rotate_origin_shift_origin_rotate_unwind_rotate', phase_low: float = -3.14, phase_up: float = 3.14, k: int = 3, include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, max_workers=1, ex=None, **kwargs) DataFrame
Interpolate target sweep phase vs. frequency data and add interpolating splines to
propertiesattribute- Parameters:
phase_low (float) – Lower bound of phase to use for interpolation. Defaults to -pi
phase_up (float) – Upper bound of phase to use for interpolation. Defaults to +pi
k (int) – Degree of polynomials to use for interpolation. Defaults to degree 3.
- phase_to_f(prefix: str | list[str] = 'mismatch_rotate_origin_shift_origin_rotate_unwind_rotate', spline_col: str = 'mismatch_rotate_origin_shift_origin_rotate_unwind_rotate', phase_bounds: float = 0.2, k: int = 3, include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, max_workers=1, ex=None, **kwargs) DataFrame
Convert timestream phase data to frequency using target sweep phase vs. frequency interpolating spline
- Parameters:
prefix (str | list[str]) – Prefix of phase data to convert to frequency
spline_col (str) – Prefix of phase data used to construct spline
phase_bounds (float) – Amount to add to min and max timestream phase to determine phase vs. frequency spline bounds (i.e.,
phase_low = min_stream_phase - phase_bounds&phase_up = max_stream_phase + phase_bounds)k (int) – Order of polynomials to use for phase vs. frequency spline
include (int | list[int] | None, optional) – Detector(s) for which to perform calculation
exclude (int | list[int] | None, optional) – Detector(s) for which not to perform calculation
recalc (bool) – Whether to recalculate if data is already in
dataDataFrame. Defaults to Falsemax_workers (int) – Number of processor cores to use for calculation. Defaults to 1.
- property pickle_dir: str
Directory where pickle files should be saved. Create if it does not already exist.
- property properties: DataFrame
- properties_histogram(col_name, bins=None, mad_filter=True, num_mads=10, filter_exprs=[], recalc=False, **kwargs)
Calculate histogram of a detector property
- Parameters:
col_name (str) – Name of property
bins (int | None, optional) – Number of bins to use for histogram. Defaults to number of tones used in histogram divided by five
mad_filter (bool, optional) – Whether to use the median absolute deviation (MAD) to filter outliers before creating histogram. Defaults to True
num_mads (int, optional) – Number of MADs data must be within to keep. Defaults to 10
recalc (bool, optional) – Whether to recalculate histogram if data already exists. Defaults to False
- Returns:
Polars DataFrame with histogram counts and bin edges
- Return type:
return (pl.DataFrame)
- properties_histogram_plot(col_name, plot_median=None, xlabel='', title='', filter_exprs=[], save_fig: bool | None = None, figs_per_file: int | None = None, overwrite: bool | None = None, save_dir: str | Path | None = None, save_name: str = None, save_fmt: Literal['png', 'jpeg', 'pdf'] | None = None, return_fig=True, return_df=False, df=None, by=None, **kwargs)
Plot histogram of a detector property
- Parameters:
col_name (str) – Name of property
plot_median (bool) – Whether to plot a vertical line at the median
- Returns:
Holoviews histogram figure
- Return type:
return (hv.Histogram | hv.Overlay)
ccatkidlib.analysis.core.network module
- class ccatkidlib.analysis.core.network.Network(com_to: str, cfg_path: str = '/home/docs/checkouts/readthedocs.org/user_builds/ccatkidlib/envs/latest/lib/python3.12/site-packages/ccatkidlib/analysis/analysis_config.yaml', dets: int | list[int] = -1, noise_tones: int | list[int] | None = None, cable_delay: float | None = None, detectors: list[Detector] | None = None, sess_ids: str | list[str] | None = None, include_streams: bool = True, include_targs: bool = False, analysis_cfg: dict | None = None, viz_cfg: dict | None = None, **kwargs)
Bases:
objectClass representing a single network of a kinetic inductance detector (KID) array.
Initialize Network object by creating Detector objects and loading them into a Polars DataFrame with the specified data columns
Note
Either the full data path must be provided via the
data_pathkey word argument or both the timestamp and data type must be provided via thedata_typeandtimestampkey word arguments respectively- Parameters:
com_to (str) – Drone that took the data. In form ‘Board.Drone’
analysis_cfg (str, optional) – Path to analysis config. Defaults to analysis config in ccatkidlib/ccatkidlib/analysis directory.
detectors (list[ccatkidlib.analysis.detector.Detector]) – List of Detector objects
sess_ids (list[str]) – List of session IDs to load target sweeps/timestreams from
include_streams (bool) – Whether to load timestreams (will also load associated target sweeps)
include_targs (bool) – Whether to load target sweeps (will load all target sweeps, even those without an associated timestream)
**kwargs – Key word arguments for finding data file. See below:
root_data_dir (str, optional) – Root directory where data is stored. Defaults to that specified in analysis config
data_dir (str, optional) – Directory where data is stored
dates (str, list[str], optional) – Date data was taken
- Raises:
ValueError – If multiple data files are specified with differing file types or timestamps
FileNotFoundError – If any data files cannot be found
- add_columns(data_cols: str | list[str], max_workers: int = 1, ex=None) DataFrame
Add columns to the Network.data DataFrame using fields from the ext_cfg or drone_cfg
- Parameters:
data_cols (str, list[str]) – List of data column names to add. Names must exactly match a field in the ext_cfg or drone_cfg
max_workers (int, optional) – Maximum number of CPU cores to use. Defaults to 1.
- combine_properties(data_cols=[])
- property fig_dir: str
Directory where figures should be saved. Create if it does not already exist.
- property pickle_dir: str
Directory where pickle files should be saved. Create if it does not already exist.
- plot(func, data_type, *args, data_cols=[], filter_exprs=[], return_df=False, save_fig: bool | None = None, overwrite: bool | None = None, save_name: str = None, overlay_cols: str | list[str] = None, **kwargs)
- Parameters:
func (str)
data_type (str)
ccatkidlib.analysis.core.sweep module
- class ccatkidlib.analysis.core.sweep.Sweep(com_to: str, **kwargs)
Bases:
DataClass representing a RFSoC frequency sweep taken using the ccatkidlib data acquisition. Sub-classes the Data base class to implement data loading and plotting.
- Parameters:
com_to – RFSoC drone that took the frequency sweep
**kwargs** – Keyword arguments of the base Data class
- IQ_plot(prefix: str = '', y_prefix: str | None = None, projection: Literal['IQ', 'polar'] = 'IQ', max_IQ_sliver: bool = False, grouping: Literal['by', 'groupby'] = 'groupby', include: int | list[int] | None = None, exclude: int | list[int] | None = None, filter_exprs: list[Expr] = [], return_df: bool = False, save_fig: bool = True, **kwargs) Overlay | HoloMap | DynamicMap | tuple[Overlay | HoloMap | DynamicMap, DataFrame] | tuple[DataFrame, str | None]
Plot the quadrature \(Q\) versus the in-phase \(I\) data of the sweep if projection is ‘IQ’. If projection is ‘polar’, plot the magnitude \(|S_{21}| = \sqrt{I^2 + Q^2}\) versus phase \(\arctan{Q/I}\) data of the sweep.
- Parameters:
prefix – Prefix of data to plot on the x-axis (either \(I\) or the phase data). Will also use as the prefix of the y-axis data if y_prefix not specified.
y_prefix – Prefix of data to plot on the y-axis (either \(Q\) or the magnitude data).
projection – Whether to plot data in polar (projection = ‘polar’) or cartesian (projection = ‘IQ’) coordinates
max_IQ_sliver
grouping – How to handle plotting sweeps for multiple tones. ‘by’ will overlay all plots whereas ‘groupby’ will allow scrubbing through sweep plots of individual tones
include – List of tones to plot
exclude – List of tones to not plot
filter_exprs – List of Polars Expr to use for filtering Polars DataFrame used for plotting
return_df – Whether to return the Polars DataFrame that was used to create figure(s)
save_fig – Whether to save figure. Defaults to that specified in viz configuration file
**kwargs**
- Returns:
Will return Holoviews figure of sweep data if
return_figis True. Will also return Polars DataFrame used to create figure ifreturn_dfis True. Ifreturn_figis False, will return DataFrame and column names to group data by
- property data: DataFrame
Polars DataFrame with sweep data. Load data if it is not already loaded. Can only be set with Polars DataFrame or LazyFrame objects (or None)
- property det_f: Series
KID frequencies identified by ccatkidlib
find_detectorsortune_tone_placementdata acquisition methodsImportant
The identified detector frequencies are NOT necessarily the same as the tone frequencies of the sweep!
- Returns:
Polars Series of identified detector frequencies
- Raises:
FileNotFoundError – If unable to load NumPy file of detector frequencies
- f(include: int | list[int] | None = None, exclude: int | list[int] | None = None) DataFrame
Frequency data of the sweep
- Parameters:
include – List of tones to include
exclude – List of tones to exclude
- Returns:
Polars **DataFrame* with frequency data
- mag_plot(prefix: str = '', grouping: Literal['by', 'groupby'] = 'groupby', include: int | list[int] | None = None, exclude: int | list[int] | None = None, filter_exprs: list[Expr] = [], return_df: bool = False, save_fig: bool | None = None, **kwargs) Overlay | HoloMap | DynamicMap | tuple[Overlay | HoloMap | DynamicMap, DataFrame] | tuple[DataFrame, str | None]
Plot magnitude \(|S_{21}| = \sqrt{I^2 + Q^2}\) versus frequency data of the sweep
- Parameters:
prefix – Prefix of the magnitude data
grouping – How to handle plotting sweeps for multiple tones. ‘by’ will overlay all plots whereas ‘groupby’ will allow scrubbing through sweep plots of individual tones
include – List of tones to plot
exclude – List of tones to not plot
filter_exprs – List of Polars Expr to use for filtering Polars DataFrame used for plotting
return_df – Whether to return the Polars DataFrame that was used to create figure(s)
save_fig – Whether to save figure. Defaults to that specified in viz configuration file
**kwargs**
- Returns:
Will return Holoviews figure of sweep data if
return_figis True. Will also return Polars DataFrame used to create figure ifreturn_dfis True. Ifreturn_figis False, will return DataFrame and column names to group data by
- phase_plot(prefix: str = '', grouping: Literal['by', 'groupby'] = 'groupby', include: int | list[int] | None = None, exclude: int | list[int] | None = None, filter_exprs: list[Expr] = [], return_df: bool = False, save_fig: bool | None = None, **kwargs) Overlay | HoloMap | DynamicMap | tuple[Overlay | HoloMap | DynamicMap, DataFrame] | tuple[DataFrame, str | None]
Plot the phase \(\arctan{Q/I}\) versus frequency data of the sweep
- Parameters:
prefix – Prefix of the phase data
grouping – How to handle plotting sweeps for multiple tones. ‘by’ will overlay all plots whereas ‘groupby’ will allow scrubbing through sweep plots of individual tones
include – List of tones to plot
exclude – List of tones to not plot
filter_exprs – List of Polars Expr to use for filtering Polars DataFrame used for plotting
return_df – Whether to return the Polars DataFrame that was used to create figure(s)
save_fig – Whether to save figure. Defaults to that specified in viz configuration file
**kwargs**
- Returns:
Will return Holoviews figure of sweep data if
return_figis True. Will also return Polars DataFrame used to create figure ifreturn_dfis True. Ifreturn_figis False, will return DataFrame and column names to group data by
- plot(x_dim: str, y_dim: str, x_prefix: str = '', y_prefix: str = '', xlabel: str | None = None, ylabel: str | None = None, grouping: Literal['by', 'groupby'] = 'groupby', include: int | list[int] | None = None, exclude: int | list[int] | None = None, plot_opts: Options | list[Options] | None = None, filter_exprs: list[Expr] = [], return_df: bool = False, save_fig: bool | None = None, figs_per_file: int | None = None, overwrite: bool | None = None, save_dir: str | Path | None = None, save_name: str | None = None, save_fmt: Literal['png', 'jpeg', 'pdf'] | None = None, return_fig: bool = True, df: DataFrame | None = None, by: str | list[str] | None = None, area_df: DataFrame | None = None, **kwargs) Overlay | HoloMap | DynamicMap | tuple[Overlay | HoloMap | DynamicMap, DataFrame] | tuple[DataFrame, str | None]
Plot sweep data and tones
- Parameters:
x_dim – Name of data to plot on x-axis without prefixes or tone suffix (e.g., ‘f’)
y_dim – Name of data to plot on y-axis without prefixes or tone suffix (e.g., ‘mag’)
x_prefix – Prefix of data to plot on x-axis
y_prefix – Prefix of data to plot on y-axis
xlabel – Label of x-axis
ylabel – Label of y-axis
grouping – How to handle plotting sweeps for multiple tones. ‘by’ will overlay all plots whereas ‘groupby’ will allow scrubbing through sweep plots of individual tones
include – List of tones to plot
exclude – List of tones to not plot
plot_opts – Holoviews Options to apply to figure(s)
filter_exprs – List of Polars Expr to use for filtering Polars DataFrame used for plotting
return_df – Whether to return the Polars DataFrame that was used to create figure(s)
save_fig – Whether to save figure. Defaults to that specified in viz configuration file
figs_per_file – Number of figures to save in a single file. Will make a \(\sqrt{\text{figs_per_file}} \times \sqrt{\text{figs_per_file}}\) grid of figures. Defaults to that specified in viz configuration file
overwrite – Whether to overwrite figure files that already exist. Defaults to that specified in viz configuration file
save_dir – Directory where figure should be saved. Defaults to
fig_dirattributesave_name – Save name of file. Will always append sweep
timestampto the end of file name. Defaults to that specified in viz configuration filesave_fmt – Format to save figure as. Defaults to that specified in viz configuration file
return_fig – Whether to return Holoviews figure
df – Polars DataFrame to use for creating figure(s). Must have the following columns: ‘sample’, ‘det’,
x_dim,y_dimby – List of column names to group data by
area_df – Polars DataFrame specifying which samples should be used to create Holoviews Area plot. Must have a ‘det’ column and a ‘area_sample’ column with lists of samples to use for area plots for each tone
**kwargs** – Keyword arguments to supply to the Holoviews
hvplotcall(s) that create the figure(s)
- Returns:
Will return Holoviews figure of sweep data if
return_figis True. Will also return Polars DataFrame used to create figure ifreturn_dfis True. Ifreturn_figis False, will return DataFrame and column names to group data by
ccatkidlib.analysis.core.target module
- class ccatkidlib.analysis.core.target.Target(com_to: str, tones: int | list[int] | None = None, noise_tones: int | list[int] | None = None, **kwargs)
Bases:
SweepClass representing a target sweep taken with a Radio Frequency System on a Chip (RFSoC)
Subclasses Sweep.
Subclass of Sweep with additional arguments
- Parameters:
tones (int | list[int] | None, optional) – Which tones to load. None for loading all data without splitting into individual tones. -1 for all data split into individual tones. Defaults to None
- property cable_delay: dict | None
Get the cable delay of the RF chain using the phase data
- Returns:
The cable delay in nanoseconds
- Return type:
float
- property data
Polars DataFrame with sweep data. Load data if it is not already loaded. Can only be set with Polars DataFrame or LazyFrame objects (or None)
- join(other, in_place=False)
Join two
Dataobjects into a single object- Parameters:
other –
Dataobject to join within_place – Whether to perform join in-place (will modify this
Dataobject) or to create newDataobject
- Returns:
A reference to this
Dataobject ifin_placeis True, otherwise the newly createdDataobject
- property properties
ccatkidlib.analysis.core.timestream module
Module for analyzing kinetic inductance detector (KID) time ordered (timestream) data
- Authors:
Darshan Patel <dp649@cornell.edu>
- class ccatkidlib.analysis.core.timestream.Timestream(com_to, tones: int | list[int] = -1, noise_tones: int | list[int] | None = None, start=0, end=-1, **kwargs)
Bases:
DataClass representing a timestream taken with a radio frequency system on a chip (RFSoC)
- tones
List of tones with loaded timestreams
- Type:
list[int]
- start
Start time of loaded timestreams in seconds (0 seconds is beginning of timestream)
- Type:
float
- end
End time of loaded timestreams in seconds (relative to start time)
- Type:
float
- packet_counts
List of packet numbers of loaded timestreams
- Type:
list[int]
- data
Polars DataFrame with loaded (and transformed) timestream data
- Type:
pl.DataFrame
- properties
Polars DataFrame with timestream properties (a ‘property’ has one value per tone)
- Type:
pl.DataFrame
- sampling_freq
Sampling frequency of timestream data in Hz
- Type:
float
Constructor for Timestream.
- Parameters:
com_to (str) – Which board and drone were used to take the timestreams (in form
bid.drid)tones (list) – List of tones for which to load timestreams
start (float) – Start time in seconds (0 seconds is beginning of timestream)
end (float) – End time in seconds (relative to start time, -1 for no end time)
analysis_cfg (str) – File path of analysis configuration file. Defaults to analysis configuration file in ccatkidlib/analysis directory.
- IQ_plot(prefix: str = '', projection='IQ', grouping='groupby', datashade=True, include: int | list[int] | None = None, exclude: int | list[int] | None = None, return_df=False, save_fig: bool | None = None, overwrite: bool | None = None, **kwargs)
- property data
Load timestream I, Q data.
- dt()
- fft(prefix: str | list[str] = '', col_name: str = 'phase', sampling_freq=None, include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, max_workers: int = 1, ex=None) DataFrame
- fft_plot(col_name, prefix: str = '', grouping='groupby', include: int | list[int] | None = None, exclude: int | list[int] | None = None, return_df=False, save_fig: bool | None = None, overwrite: bool | None = None, **kwargs)
- join(other, in_place=False)
Join two
Dataobjects into a single object- Parameters:
other –
Dataobject to join within_place – Whether to perform join in-place (will modify this
Dataobject) or to create newDataobject
- Returns:
A reference to this
Dataobject ifin_placeis True, otherwise the newly createdDataobject
- load_frame(frame, start_time: float, time_precision: int = 100000000.0, mask: list[bool] = None) tuple[list[float], list[list[float]], list[list[float]]]
- mag_plot(x_prefix: str = '', y_prefix: str = '', grouping='groupby', datashade=None, include: int | list[int] | None = None, exclude: int | list[int] | None = None, return_df=False, save_fig: bool | None = None, overwrite: bool | None = None, **kwargs)
- phase_plot(x_prefix: str = '', y_prefix: str = '', grouping='groupby', datashade=None, include: int | list[int] | None = None, exclude: int | list[int] | None = None, return_df=False, save_fig: bool | None = None, overwrite: bool | None = None, **kwargs)
- plot(x_dim, y_dim, x_prefix: str = '', y_prefix: str = '', xlabel: str = '', ylabel: str = '', grouping: str = 'groupby', include: int | list[int] | None = None, exclude: int | list[int] | None = None, plot_opts=None, filter_exprs=[], unpivot_x=True, return_df=False, save_fig: bool | None = None, figs_per_file: int | None = None, overwrite: bool | None = None, save_dir: str | Path | None = None, save_name: str = None, save_fmt: Literal['png', 'jpeg', 'pdf'] | None = None, return_fig=True, df: DataFrame | None = None, by: str | list[str] | None = None, **kwargs)
- property properties
- psd(prefix: str | list[str] = '', col_name: str = 'phase', window='hann', nperseg=None, detrend=False, average='mean', include: int | list[int] | None = None, exclude: int | list[int] | None = None, recalc: bool = False, max_workers: int = 1, ex=None) DataFrame
- psd_plot(col_name, prefix: str = '', grouping='groupby', include: int | list[int] | None = None, exclude: int | list[int] | None = None, return_df=False, save_fig: bool | None = None, overwrite: bool | None = None, **kwargs)
- property sampling_freq
- stream_plot(col_name: str, prefix: str = '', time_col='t', grouping='groupby', datashade=None, include: int | list[int] | None = None, exclude: int | list[int] | None = None, return_df=False, save_fig: bool | None = None, overwrite: bool | None = None, **kwargs)
Plot the specified data column as a function of time
- Parameters:
col_name (str) – Name of data column (e.g., I, Q, mag, etc.)
prefix (str, optional) – Defaults to “”
return_df (bool) – Whether to return the Polars DataFrame that was used to create the plot. Defaults to False
include (int | list[int] | None, optional) – Defaults to None
exclude (int | list[int] | None, optional) – Defaults to None
- Return type:
return (hv.NdOverlay | tuple[hv.NdOverlay, pl.DataFrame])
- t()
ccatkidlib.analysis.core.vna module
- class ccatkidlib.analysis.core.vna.VNA(com_to: str, **kwargs)
Bases:
SweepClass representing a vector network analyzer (VNA) esque sweep taken with a Radio Frequency System on a Chip (RFSoC).
Subclasses the Sweep class.
- cable_delay
Cable delay of the RF chain in nanoseconds
- Type:
float
- Parameters:
com_to – RFSoC drone that took the frequency sweep
**kwargs** – Keyword arguments of the base Data class
- property cable_delay: float
Get the cable delay of the RF chain using the phase data
- Returns:
The cable delay in nanoseconds
- Return type:
float
- filter_det_f(win: int = 3, stitch_phase: bool = True) tuple[ndarray, ndarray]
Filters out detectors found by find_resonators that are likely fake (noise or otherwise) based on the slope of the phase around the found frequency
Original implementation by Ben Keller, modified to work with Numba. :param win: The window size with which to fit the phase around the found frequency. :type win: int :param stitch_phase: Whether to use stitched phase data :type stitch_phase: bool
- Returns:
Frequencies of likely real detectors, frequencies of likely fake detectors.
- Return type:
tuple[np.ndarray, np.ndarray]
- stitch_mag(stitch_percent: float = 10.0, med_win: int = 3, recalc: bool = False) LazyFrame
Stitch the VNA sweep mag data to remove discontinuites at bin edges
Note
Only one of
includeandexcludearguments can be specified. If neither are specified, returns data for all tones.- Parameters:
include (int | list[int], optional) – List of tones to include. Defaults to None
exclude (int | list[int], optional) – List of tones to exclude. Defaults to None
- Returns:
Polars DataFrame with specified data columns
- Return type:
polars.lazyframe.frame.LazyFrame
- stitch_phase(threshold: float = 5.969026041820607, stitch_percent: float = 10.0, recalc: bool = False) LazyFrame
Stitch the VNA sweep phase data to remove discontinuites from wrapping and bin edges
Note
Only one of
includeandexcludearguments can be specified. If neither are specified, returns data for all tones.- Parameters:
include (int | list[int], optional) – List of tones to include. Defaults to None
exclude (int | list[int], optional) – List of tones to exclude. Defaults to None
- Returns:
Polars DataFrame with specified data columns
- Return type:
polars.lazyframe.frame.LazyFrame
- ccatkidlib.analysis.core.vna.filter_det_f(f, phase, det_f, win)