Dimensionality reduction

Standalone dimensionality-reduction functions. Each one is also exposed as a Dataset method of the same name. All four support a circular=True mode for angular features (e.g. dihedral angles), which switches to a distance/kernel that correctly handles the +-pi wrap-around instead of treating angles as plain linear numbers.

PCA

Principal Component Analysis: a fast, linear projection onto the directions of maximum variance. Good default first pass, and the only one of the four that scales comfortably to large datasets.

RepLikCompare.dimensionality_reduction.compute_pca.compute_pca(df, feature_cols=None, n_components=2, plot=True, s=5, color='C0', edgecolors='black', context='notebook', figsize=(10, 5), ax=None)[source]

Reduce the dimensionality of the data using PCA.

Parameters:

  • df (pandas.DataFrame) – Data to reduce. If feature_cols is given, only those columns are used to fit PCA (the returned dataframe keeps ALL of df’s original columns, plus the PCA components) – use this whenever df also carries metadata columns (sim_name, replica, frame, …) that must not be fed to PCA itself. If feature_cols is None (default, backward-compatible), df is used as-is – every column must be numeric, or PCA.fit_transform will raise.

  • feature_cols (list of str, optional) – Column names in df to actually reduce. Strongly recommended whenever df has metadata columns beyond the raw numeric features. Default None (use all of df).

  • n_components (int, optional) – Number of components to keep. The default is 2.

  • plot (bool, optional) – Plot the two first components. The default is True.

  • s (int, optional) – Marker size for the scatter plot. The default is 5.

  • color (str, optional) – Marker color for the scatter plot. The default is “C0”.

  • context (str, optional) – Seaborn plotting context. The default is “notebook”.

  • figsize (tuple, optional) – Figure size. The default is (10, 5).

  • ax (matplotlib.axes._subplots.AxesSubplot, optional) – Axes on which to plot. The default is None.

Returns:

  • new_df (pandas.DataFrame) – df (all original columns preserved) with the PCA components added (columns “PC1”, “PC2”, …).

  • fig (matplotlib.figure.Figure or None) – Figure containing the plot (if plot=True) – always retrievable as an object for further tweaking, e.g. fig.savefig(…).

  • pca (sklearn.decomposition.PCA) – The fitted PCA object (e.g. for pca.explained_variance_ratio_, pca.components_).

UMAP

A nonlinear, graph-based embedding that tends to preserve both local neighborhoods and some global structure. Generally the best-looking 2D/3D embedding for exploratory visualization, at a higher computational cost than PCA.

RepLikCompare.dimensionality_reduction.compute_umap.compute_umap(df, feature_cols=None, n_neighbors=15, min_dist=0.1, n_components=2, metric='euclidean', circular=False, random_state=None, plot=True, s=5, color='C0', edgecolors='black', context='notebook', figsize=(10, 5), ax=None, verbose=True)[source]

Reduce the dimensionality of the data using UMAP.

Parameters:

  • df (pandas.DataFrame) – Data to reduce. If feature_cols is given, only those columns are used to fit UMAP (the returned dataframe keeps ALL of df’s original columns, plus the UMAP components) – use this whenever df also carries metadata columns (sim_name, replica, frame, …) that must not be fed to UMAP itself. If feature_cols is None (default, backward-compatible), df is used as-is – every column must be numeric.

  • feature_cols (list of str, optional) – Column names in df to actually reduce. Default None (use all of df).

  • n_neighbors (int, optional) – Number of neighbors to consider for each point. The default is 15.

  • min_dist (float, optional) – Minimum distance between embedded points. The default is 0.1.

  • n_components (int, optional) – Number of dimensions of the embedded space. The default is 2.

  • metric (str, optional) – Metric to use. Ignored if circular is True. The default is “euclidean”.

  • circular (bool, optional) – If True, use a circular-aware metric suited for angular features (based on the unit vector representation of the angles, in radians). This overrides the metric argument. The default is False.

  • random_state (int, optional) – Random seed. The default is None.

  • plot (bool, optional) – Plot the two first components. The default is True.

  • s (int, optional) – Marker size for the scatter plot. The default is 5.

  • color (str, optional) – Marker color for the scatter plot. The default is “C0”.

  • context (str, optional) – Seaborn plotting context. The default is “notebook”.

  • figsize (tuple, optional) – Figure size. The default is (10, 5).

  • ax (matplotlib.axes._subplots.AxesSubplot, optional) – Axes on which to plot. The default is None.

Returns:

  • new_df (pandas.DataFrame) – df (all original columns preserved) with the UMAP components added (columns “UMAP1”, “UMAP2”, …).

  • fig (matplotlib.figure.Figure or None) – Figure containing the plot (if plot=True) – always retrievable as an object for further tweaking, e.g. fig.savefig(…).

  • umap_model (umap.UMAP) – The fitted UMAP object.

t-SNE

Another nonlinear embedding, focused on preserving local neighborhoods (at the expense of global distances/structure). Often produces tighter, more visually separated clusters than UMAP, but inter-cluster distances shouldn’t be over-interpreted.

RepLikCompare.dimensionality_reduction.compute_tsne.compute_tsne(df, feature_cols=None, perplexity=30, n_components=2, metric='euclidean', circular=False, learning_rate='auto', random_state=None, plot=True, s=5, color='C0', edgecolors='black', context='notebook', figsize=(10, 5), ax=None, verbose=1)[source]

Reduce the dimensionality of the data using t-SNE.

Parameters:

  • df (pandas.DataFrame) – Data to reduce. If feature_cols is given, only those columns are used to fit t-SNE (the returned dataframe keeps ALL of df’s original columns, plus the t-SNE components) – use this whenever df also carries metadata columns (sim_name, replica, frame, …) that must not be fed to t-SNE itself. If feature_cols is None (default, backward-compatible), df is used as-is – every column must be numeric.

  • feature_cols (list of str, optional) – Column names in df to actually reduce. Default None (use all of df).

  • perplexity (float, optional) – Perplexity of the t-SNE embedding. The default is 30.

  • n_components (int, optional) – Number of dimensions of the embedded space. The default is 2.

  • metric (str, optional) – Metric to use. Ignored if circular is True. The default is “euclidean”.

  • circular (bool, optional) – If True, use a circular-aware metric suited for angular features (based on the unit vector representation of the angles, in radians). This overrides the metric argument. The default is False.

  • learning_rate (float or str, optional) – Learning rate. The default is “auto”.

  • random_state (int, optional) – Random seed. The default is None.

  • plot (bool, optional) – Plot the two first components. The default is True.

  • s (int, optional) – Marker size for the scatter plot. The default is 5.

  • color (str, optional) – Marker color for the scatter plot. The default is “C0”.

  • context (str, optional) – Seaborn plotting context. The default is “notebook”.

  • figsize (tuple, optional) – Figure size. The default is (10, 5).

  • ax (matplotlib.axes._subplots.AxesSubplot, optional) – Axes on which to plot. The default is None.

Returns:

  • new_df (pandas.DataFrame) – df (all original columns preserved) with the t-SNE components added (columns “TSNE1”, “TSNE2”, …).

  • fig (matplotlib.figure.Figure or None) – Figure containing the plot (if plot=True) – always retrievable as an object for further tweaking, e.g. fig.savefig(…).

  • tsne (sklearn.manifold.TSNE) – The fitted TSNE object.

Kernel PCA

A nonlinear generalization of PCA using the kernel trick (e.g. an RBF kernel); a middle ground between plain PCA and UMAP/t-SNE.

RepLikCompare.dimensionality_reduction.compute_kpca.compute_kpca(df, feature_cols=None, circular=False, n_components=10, kernel='poly', gamma=None, plot=True, s=5, color='C0', context='notebook', figsize=(10, 5), edgecolors='black', ax=None)[source]

Reduce the dimensionality of the data using Kernel PCA (KPCA).

Parameters:

  • df (pandas.DataFrame) – Data to reduce. If feature_cols is given, only those columns are used to fit KPCA (the returned dataframe keeps ALL of df’s original columns, plus the KPCA components) – use this whenever df also carries metadata columns (sim_name, replica, frame, …) that must not be fed to KPCA itself. If feature_cols is None (default, backward-compatible), df is used as-is – every column must be numeric.

  • feature_cols (list of str, optional) – Column names in df to actually reduce. Default None (use all of df).

  • circular (bool, optional) – If True, use a custom similarity kernel suited for angular features (based on the unit vector representation of the angles, in radians). This overrides the kernel argument. The default is False.

  • n_components (int, optional) – Number of components to keep. The default is 10.

  • kernel (str, optional) – Kernel used for KPCA, as in the scikit-learn implementation. Ignored if circular is True. The default is “poly”.

  • gamma (float, optional) – Kernel coefficient. If None and circular is True, it is set to 1 / number of features. The default is None.

  • plot (bool, optional) – Plot the two first components. The default is True.

  • s (int, optional) – Marker size for the scatter plot. The default is 5.

  • color (str, optional) – Marker color for the scatter plot. The default is “C0”.

  • context (str, optional) – Seaborn plotting context. The default is “notebook”.

  • figsize (tuple, optional) – Figure size. The default is (10, 5).

  • ax (matplotlib.axes._subplots.AxesSubplot, optional) – Axes on which to plot. The default is None.

Returns:

  • new_df (pandas.DataFrame) – df (all original columns preserved) with the KPCA components added (columns “KPC1”, “KPC2”, …).

  • fig (matplotlib.figure.Figure or None) – Figure containing the plot (if plot=True) – always retrievable as an object for further tweaking, e.g. fig.savefig(…).

  • kpca (sklearn.decomposition.KernelPCA) – The fitted KernelPCA object.

Note

KPCA computes an N x N kernel matrix (N = number of rows) – O(N^2) memory and O(N^3) time for the eigendecomposition. This becomes slow or memory-heavy well before N reaches tens of thousands of rows; subsample df/feature_cols first (e.g. df.sample(n=2000)) on large trajectories.

Notebook example