# Package peppy Documentation

## Package Overview

The `peppy` package provides a Python interface for working with Portable Encapsulated Projects (PEPs). A PEP is a standardized format for organizing metadata for biological samples and sample-intensive data.

### Key Features

- **Project Management**: Create and manage collections of samples with metadata
- **Sample Access**: Retrieve individual samples and their attributes
- **Amendments**: Activate different project configurations
- **Validation**: Validate projects against schemas

### Installation

```bash
pip install peppy
```

### Quick Example

```python
from peppy import Project

# Initialize with a project config file
prj = Project(cfg="ngs.yaml")

# Access samples
samples = prj.samples
```

## API Reference

### Project Class

The main class for working with PEPs:

<a id="peppy.Project"></a>

### `Project`

```python
class Project(cfg: str = None, amendments: Union[str, Iterable[str]] = None, sample_table_index: Union[str, Iterable[str]] = None, subsample_table_index: Union[str, Iterable[str]] = None, defer_samples_creation: bool = False)
```

Bases: `MutableMapping`

A class to model a Project (collection of samples and metadata).

:param str cfg: Project config file (YAML) or sample table (CSV/TSV)
    with one row per sample to constitute project
:param str | Iterable[str] sample_table_index: name of the columns to set
    the sample_table index to
:param str | Iterable[str] subsample_table_index: name of the columns to set
    the subsample_table index to
:param str | Iterable[str] amendments: names of the amendments to activate
:param Iterable[str] amendments: amendments to use within configuration file
:param bool defer_samples_creation: whether the sample creation should be skipped

:Example:

```python
from peppy import Project
prj = Project(cfg="ngs.yaml")
samples = prj.samples
```

#### Attributes

- <a id="peppy.Project.progressbar"></a>**`progressbar`**
- <a id="peppy.Project.sst_index"></a>**`sst_index`**
- <a id="peppy.Project.st_index"></a>**`st_index`**

#### Properties

<a id="peppy.Project.amendments"></a>

##### `amendments`

```python
@property
def amendments
```

Return currently active list of amendments or None if none was activated

:return Iterable[str]: a list of currently active amendment names

<a id="peppy.Project.config"></a>

##### `config`

```python
@property
def config
```

Get the config mapping

:return Mapping: config. May be formatted to comply with the most
    recent version specifications

<a id="peppy.Project.config_file"></a>

##### `config_file`

```python
@property
def config_file
```

Get the config file path

:return str: path to the config file

<a id="peppy.Project.description"></a>

##### `description`

```python
@property
def description
```

<a id="peppy.Project.is_sample_table_large"></a>

##### `is_sample_table_large`

```python
@property
def is_sample_table_large
```

<a id="peppy.Project.list_amendments"></a>

##### `list_amendments`

```python
@property
def list_amendments
```

Return a list of available amendments or None if not declared

:return Iterable[str]: a list of available amendment names

<a id="peppy.Project.name"></a>

##### `name`

```python
@property
def name
```

<a id="peppy.Project.pep_version"></a>

##### `pep_version`

```python
@property
def pep_version
```

The declared PEP version string

It is validated to make sure it is a valid PEP version string

:raise InvalidConfigFileException: in case of invalid PEP version
:return str: PEP version string

<a id="peppy.Project.sample_name_colname"></a>

##### `sample_name_colname`

```python
@property
def sample_name_colname
```

**Deprecated, please use `Project.sample_table_index` instead**

Name of the effective sample name containing column in the sample table.

It is "sample_name" by default, but when it's missing it could be
replaced by the selected sample table index, defined on the
object instantiation stage.

:return str: name of the column that consist of sample identifiers

<a id="peppy.Project.sample_table"></a>

##### `sample_table`

```python
@property
def sample_table
```

Get sample table. If any sample edits were performed,
it will be re-generated

:return pandas.DataFrame: a data frame with current samples attributes

<a id="peppy.Project.sample_table_index"></a>

##### `sample_table_index`

```python
@property
def sample_table_index
```

The effective sample table index.

It is `sample_name` by default, but could be overwritten by the selected sample table index,
defined on the object instantiation stage or in the project configuration file
via `sample_table_index` field.

That's the sample table index selection priority order:

1. Constructor specified
2. Config specified
3. Deafult: `sample_table`

:return str: name of the column that consist of sample identifiers

<a id="peppy.Project.samples"></a>

##### `samples`

```python
@property
def samples
```

Generic/base Sample instance for each of this Project's samples.

:return Iterable[Sample]: Sample instance for each
    of this Project's samples

<a id="peppy.Project.subsample_table"></a>

##### `subsample_table`

```python
@property
def subsample_table
```

Get subsample table

:return pandas.DataFrame: a data frame with subsample attributes

<a id="peppy.Project.subsample_table_index"></a>

##### `subsample_table_index`

```python
@property
def subsample_table_index
```

The effective subsample table indexes.

It is `[subasample_name, sample_name]` by default,
but could be overwritten by the selected subsample table indexes,
defined on the object instantiation stage or in the project configuration file
via `subsample_table_index` field.

That's the subsample table indexes selection priority order:

1. Constructor specified
2. Config specified
3. Deafult: `[subasample_name, sample_name]`

:return List[str]: names of the columns that consist of sample and subsample identifiers

#### Class methods

<a id="peppy.Project.from_dict"></a>

##### `from_dict`

```python
@classmethod
def from_dict(pep_dictionary: dict)
```

Init a peppy project instance from a dictionary representation
of an already processed PEP.

:param Dict[Any] pep_dictionary: dict representation of the project {_config: dict,
                                                                     _samples: list | dict,
                                                                     _subsamples: list[list | dict]}

<a id="peppy.Project.from_pandas"></a>

##### `from_pandas`

```python
@classmethod
def from_pandas(samples_df: pd.DataFrame, sub_samples_df: List[pd.DataFrame] = None, config: dict = None)
```

Init a peppy project instance from a pandas Dataframe

:param samples_df: in-memory pandas DataFrame object of samples
:param sub_samples_df: in-memory list of pandas DataFrame objects of sub-samples
:param config: dict of yaml file

<a id="peppy.Project.from_pep_config"></a>

##### `from_pep_config`

```python
@classmethod
def from_pep_config(cfg: str = None, amendments: Union[str, Iterable[str]] = None, sample_table_index: Union[str, Iterable[str]] = None, subsample_table_index: Union[str, Iterable[str]] = None, defer_samples_creation: bool = False)
```

Init a peppy project instance from a yaml file

:param str cfg: Project config file (YAML) or sample table (CSV/TSV)
    with one row per sample to constitute project
:param str | Iterable[str] sample_table_index: name of the columns to set
    the sample_table index to
:param str | Iterable[str] subsample_table_index: name of the columns to set
    the subsample_table index to
:param str | Iterable[str] amendments: names of the amendments to activate
:param Iterable[str] amendments: amendments to use within configuration file
:param bool defer_samples_creation: whether the sample creation should be skipped

<a id="peppy.Project.from_pephub"></a>

##### `from_pephub`

```python
@classmethod
def from_pephub(registry_path: str) -> Project
```

Init project from pephubclient.

:param registry_path: PEPhub registry path
:return: peppy Project

<a id="peppy.Project.from_sample_yaml"></a>

##### `from_sample_yaml`

```python
@classmethod
def from_sample_yaml(yaml_file: str)
```

Init a peppy project instance from a yaml file

:param str yaml_file: path to yaml file

#### Methods

<a id="peppy.Project.__getitem__"></a>

##### `__getitem__`

```python
def __getitem__(item)
```

Fetch the value of given key.

:param hashable item: key for which to fetch value
:return object: value mapped to given key, if available
:raise KeyError: if the requested key is unmapped.

<a id="peppy.Project.__str__"></a>

##### `__str__`

```python
def __str__()
```

Representation in interpreter.

<a id="peppy.Project.activate_amendments"></a>

##### `activate_amendments`

```python
def activate_amendments(amendments)
```

Update settings based on amendment-specific values.

This method will update Project attributes, adding new values
associated with the amendments indicated, and in case of collision with
an existing key/attribute the amendments' values will be favored.

:param Iterable[str] amendments: A string with amendment
    names to be activated
:return peppy.Project: Updated Project instance
:raise TypeError: if argument to amendment parameter is null
:raise NotImplementedError: if this call is made on a project not
    created from a config file

<a id="peppy.Project.add_samples"></a>

##### `add_samples`

```python
def add_samples(samples)
```

Add list of Sample objects

:param peppy.Sample | Iterable[peppy.Sample] samples: samples to add

<a id="peppy.Project.attr_constants"></a>

##### `attr_constants`

```python
def attr_constants()
```

Update each Sample with constants declared by a Project.
If Project does not declare constants, no update occurs.

<a id="peppy.Project.attr_derive"></a>

##### `attr_derive`

```python
def attr_derive(attrs=None)
```

Set derived attributes for all Samples tied to this Project instance

<a id="peppy.Project.attr_imply"></a>

##### `attr_imply`

```python
def attr_imply()
```

Infer value for additional field(s) from other field(s).

Add columns/fields to the sample based on values in those already-set
that the sample's project defines as indicative of implications for
additional data elements for the sample.

<a id="peppy.Project.attr_merge"></a>

##### `attr_merge`

```python
def attr_merge()
```

Merge sample subannotations (from subsample table) with
sample annotations (from sample_table)

<a id="peppy.Project.attr_remove"></a>

##### `attr_remove`

```python
def attr_remove()
```

Remove declared attributes from all samples that have them defined

<a id="peppy.Project.attr_synonyms"></a>

##### `attr_synonyms`

```python
def attr_synonyms()
```

Copy attribute values for all samples to a new one

<a id="peppy.Project.create_samples"></a>

##### `create_samples`

```python
def create_samples(modify: bool = False)
```

Populate Project with Sample objects

<a id="peppy.Project.deactivate_amendments"></a>

##### `deactivate_amendments`

```python
def deactivate_amendments()
```

Bring the original project settings back.

:return peppy.Project: Updated Project instance
:raise NotImplementedError: if this call is made on a project not
    created from a config file

<a id="peppy.Project.get_description"></a>

##### `get_description`

```python
def get_description()
```

Infer project description from config file.

The provided description has to be of class coercible to string

:return str: inferred name for project.
:raise InvalidConfigFileException: if description is not of class
    coercible to string

<a id="peppy.Project.get_sample"></a>

##### `get_sample`

```python
def get_sample(sample_name)
```

Get an individual sample object from the project.

Will raise a ValueError if the sample is not found.
In the case of multiple samples with the same name (which is not
typically allowed), a warning is raised and the first sample is returned

:param str sample_name: The name of a sample to retrieve
:raise ValueError: if there's no sample with the specified name defined
:return peppy.Sample: The requested Sample object

<a id="peppy.Project.get_samples"></a>

##### `get_samples`

```python
def get_samples(sample_names)
```

Returns a list of sample objects given a list of sample names

:param list sample_names: A list of sample names to retrieve
:return list[peppy.Sample]: A list of Sample objects

<a id="peppy.Project.infer_name"></a>

##### `infer_name`

```python
def infer_name()
```

Infer project name from config file path.

First assume the name is the folder in which the config file resides,
unless that folder is named "metadata", in which case the project name
is the parent of that folder.

:return str: inferred name for project.
:raise InvalidConfigFileException: if the project lacks both a name and
    a configuration file (no basis, then, for inference)
:raise InvalidConfigFileException: if specified Project name is invalid

<a id="peppy.Project.load_samples"></a>

##### `load_samples`

```python
def load_samples()
```

Read the sample_table and subsample_tables into dataframes
and store in the object root. The values sourced from the
project config can be overwritten by the optional arguments.

<a id="peppy.Project.modify_samples"></a>

##### `modify_samples`

```python
def modify_samples()
```

Perform any sample modifications defined in the config.

<a id="peppy.Project.parse_config_file"></a>

##### `parse_config_file`

```python
def parse_config_file(cfg_path: str = None, amendments: Iterable[str] = None)
```

Parse provided yaml config file and check required fields exist.

:param str cfg_path: path to the config file to read and parse
:param Iterable[str] amendments: Name of amendments to activate
:raises KeyError: if config file lacks required section(s)

<a id="peppy.Project.remove_samples"></a>

##### `remove_samples`

```python
def remove_samples(sample_names)
```

Remove Samples from Project

:param Iterable[str] sample_names: sample names to remove

<a id="peppy.Project.to_dict"></a>

##### `to_dict`

```python
def to_dict(extended: bool = False, orient: Literal['dict', 'list', 'series', 'split', 'tight', 'records', 'index'] = 'dict') -> dict
```

Convert the Project object to a dictionary.

:param bool extended: whether to produce complete project dict (used to reinit the project)
:param Literal orient: orientation of the returned df
:return dict: a dictionary representation of the Project object

### Sample Class

<a id="peppy.Sample"></a>

### `Sample`

```python
class Sample(series, prj=None)
```

Bases: `SimpleAttMap`

Class to model Samples based on a pandas Series.

:param Mapping | pandas.core.series.Series series: Sample's data.

#### Properties

<a id="peppy.Sample.project"></a>

##### `project`

```python
@property
def project
```

Get the project mapping

:return peppy.Project: project object the sample was created from

<a id="peppy.Sample.sample_name"></a>

##### `sample_name`

```python
@property
def sample_name
```

Get the sample's name

:return str: current sample name derived from project's st_index

#### Methods

<a id="peppy.Sample.__str__"></a>

##### `__str__`

```python
def __str__(max_attr=10)
```

Representation in interpreter.

<a id="peppy.Sample.derive_attribute"></a>

##### `derive_attribute`

```python
def derive_attribute(data_sources, attr_name)
```

Uses the template path provided in the project config section
"data_sources" to piece together an actual path by substituting
variables (encoded by "{variable}"") with sample attributes.

:param Mapping data_sources: mapping from key name (as a value in
    a cell of a tabular data structure) to, e.g., filepath
:param str attr_name: Name of sample attribute
    (equivalently, sample sheet column) specifying a derived column.
:return str: regex expansion of data source specified in configuration,
    with variable substitutions made
:raises ValueError: if argument to data_sources parameter is null/empty

<a id="peppy.Sample.get_sheet_dict"></a>

##### `get_sheet_dict`

```python
def get_sheet_dict()
```

Create a K-V pairs for items originally passed in via the sample sheet.
This is useful for summarizing; it provides a representation of the
sample that excludes things like config files and derived entries.

:return OrderedDict: mapping from name to value for data elements
    originally provided via the sample sheet (i.e., the a map-like
    representation of the instance, excluding derived items)

<a id="peppy.Sample.to_dict"></a>

##### `to_dict`

```python
def to_dict(add_prj_ref=False)
```

Serializes itself as dict object.

:param bool add_prj_ref: whether the project reference bound do the
    Sample object should be included in the YAML representation
:return dict: dict representation of this Sample

<a id="peppy.Sample.to_yaml"></a>

##### `to_yaml`

```python
def to_yaml(path: Optional[str] = None, add_prj_ref=False) -> Union[str, None]
```

Serializes itself in YAML format. Writes to file if path is provided, else returns string representation.

:param str path: A file path to write yaml to; provide this or
    the subs_folder_path, defaults to None
:param bool add_prj_ref: whether the project reference bound do the
    Sample object should be included in the YAML representation
:return str | None: returns string representation of sample yaml or None