## Package Overview

The `geofetch` package provides tools for downloading metadata and data from Gene Expression Omnibus (GEO) and Sequence Read Archive (SRA). It can convert GEO/SRA metadata into PEP format for easy integration with other PEPkit tools.

### Key Features

- **GEO/SRA Download**: Fetch metadata and raw data from NCBI repositories
- **PEP Generation**: Automatically create PEP-formatted project configs
- **Flexible Filtering**: Search and filter GEO datasets by date and criteria
- **SRA Integration**: Download and convert SRA data to FASTQ format
- **Processed Data**: Download processed data matrices from GEO

### Installation

```bash
pip install geofetch
```

### Quick Example

```python
from geofetch import Geofetcher

# Initialize geofetcher
gf = Geofetcher()

# Fetch a GEO series
gf.fetch_all(input="GSE####", name="my_project")
```

## API Reference

### Geofetcher Class

The main class for fetching data from GEO/SRA:

<a id="geofetch.Geofetcher"></a>

### `Geofetcher`

```python
class Geofetcher(name: str = '', metadata_root: str = '', metadata_folder: str = '', just_metadata: bool = False, refresh_metadata: bool = False, config_template: str | None = None, pipeline_samples: str | None = None, pipeline_project: str | None = None, skip: int = 0, acc_anno: bool = False, use_key_subset: bool = False, processed: bool = False, data_source: str = 'samples', filter: str | None = None, filter_size: str | None = None, geo_folder: str = '.', split_experiments: bool = False, bam_folder: str = '', fq_folder: str = '', sra_folder: str = '', bam_conversion: bool = False, picard_path: str = '', input: str | None = None, const_limit_project: int = 50, const_limit_discard: int = 1000, attr_limit_truncate: int = 500, max_soft_size: str = '1GB', discard_soft: bool = False, add_dotfile: bool = False, disable_progressbar: bool = False, add_convert_modifier: bool = False, opts: object | None = None, max_prefetch_size: str | int | None = None, **kwargs: object)
```

Class to download or get projects, metadata, data from GEO and SRA.

Constructor.

**Parameters:**

- `input` (*str | None*): GSE number or path to the input file. (default: `None`)
- `name` (*str*): Specify a project name. Defaults to GSE number or name of accessions file name. (default: `''`)
- `metadata_root` (*str*): Specify a parent folder location to store metadata.
  The project name will be added as a subfolder (Default: $SRAMETA:). (default: `''`)
- `metadata_folder` (*str*): Specify an absolute folder location to store metadata. No subfolder will be added.
  Overrides value of --metadata-root (Default: Not used (--metadata-root is used by default)). (default: `''`)
- `just_metadata` (*bool*): If set, don't actually run downloads, just create metadata. (default: `False`)
- `refresh_metadata` (*bool*): If set, re-download metadata even if it exists. (default: `False`)
- `config_template` (*str | None*): Project config yaml file template. (default: `None`)
- `pipeline_samples` (*str | None*): Specify one or more filepaths to SAMPLES pipeline interface yaml files.
  These will be added to the project config file to make it immediately compatible with looper.
  (Default: null). (default: `None`)
- `pipeline_project` (*str | None*): Specify one or more filepaths to PROJECT pipeline interface yaml files.
  These will be added to the project config file to make it immediately compatible with looper.
  (Default: null). (default: `None`)
- `acc_anno` (*bool*): Produce annotation sheets for each accession.
  Project combined PEP for the whole project won't be produced. (default: `False`)
- `discard_soft` (*bool*): Create project without downloading soft files on the disc. (default: `False`)
- `add_dotfile` (*bool*): Add .pep.yaml file that points .yaml PEP file. (default: `False`)
- `disable_progressbar` (*bool*): Set true to disable progressbar. (default: `False`)
- `const_limit_project` (*int*): Optional: Limit of the number of the constant sample characters
  that should not be in project yaml. (Default: 50). (default: `50`)
- `const_limit_discard` (*int*): Optional: Limit of the number of the constant sample characters
  that should not be discarded (Default: 250). (default: `1000`)
- `attr_limit_truncate` (*int*): Optional: Limit of the number of sample characters.
  Any attribute with more than X characters will truncate to the first X, where X is a number of characters
  (Default: 500). (default: `500`)
- `max_soft_size` (*str*): Optional: Max size of soft file.
  Supported input formats: 12B, 12KB, 12MB, 12GB. [Default value: 1GB]. (default: `'1GB'`)
- `processed` (*bool*): Download processed data (Default: download raw data). (default: `False`)
- `data_source` (*str*): Specifies the source of data on the GEO record to retrieve processed data,
  which may be attached to the collective series entity, or to individual samples. Allowable values are:
  samples, series or both (all). Ignored unless 'processed' flag is set. (Default: samples). (default: `'samples'`)
- `filter` (*str | None*): Filter regex for processed filenames (Default: None). Ignored unless 'processed' flag is set. (default: `None`)
- `filter_size` (*str | None*): Filter size for processed files that are stored as sample repository (Default: None).
  Works only for sample data. Supported input formats: 12B, 12KB, 12MB, 12GB.
  Ignored unless 'processed' flag is set. (default: `None`)
- `geo_folder` (*str*): Specify a location to store processed GEO files.
  Ignored unless 'processed' flag is set. (Default: $GEODATA:). (default: `'.'`)
- `split_experiments` (*bool*): Split SRR runs into individual samples. By default, SRX experiments with multiple SRR
  Runs will have a single entry in the annotation table, with each run as a separate row in the
  subannotation table. This setting instead treats each run as a separate sample [Works with raw data]. (default: `False`)
- `bam_folder` (*str*): Optional: Specify folder of bam files. Geofetch will not download sra files when
  corresponding bam files already exist. (Default: $SRABAM:) [Works with raw data]. (default: `''`)
- `fq_folder` (*str*): Optional: Specify folder of fastq files. Geofetch will not download sra files when corresponding
  fastq files already exist. (Default: $SRAFQ:) [Works with raw data]. (default: `''`)
- `use_key_subset` (*bool*): Use just the keys defined in this module when writing out metadata. [Works with raw data]. (default: `False`)
- `sra_folder` (*str*): Optional: Specify a location to store sra files. (default: `''`)
- `bam_conversion` (*bool*): Optional: set True to convert bam files [Works with raw data]. (default: `False`)
- `picard_path` (*str*): Specify a path to the picard jar, if you want to convert fastq to bam [Works with raw data]. (default: `''`)
- `add_convert_modifier` (*bool*): Add looper SRA convert modifier to config file. (default: `False`)
- `skip` (*int*): Skip some accessions. (Default: no skip). (default: `0`)
- `opts` (*object | None*): opts object [Optional]. (default: `None`)
- `max_prefetch_size` (*str | int | None*): Argument to prefetch command's --max-size option. (default: `None`)
- `kwargs` (*object*): Other values. (default: `{}`)

#### Attributes

- <a id="geofetch.Geofetcher.acc_anno"></a>**`acc_anno`**
- <a id="geofetch.Geofetcher.add_convert_modifier"></a>**`add_convert_modifier`**
- <a id="geofetch.Geofetcher.add_dotfile"></a>**`add_dotfile`**
- <a id="geofetch.Geofetcher.attr_limit_truncate"></a>**`attr_limit_truncate`**
- <a id="geofetch.Geofetcher.bam_conversion"></a>**`bam_conversion`**
- <a id="geofetch.Geofetcher.bam_folder"></a>**`bam_folder`**
- <a id="geofetch.Geofetcher.config_template"></a>**`config_template`**
- <a id="geofetch.Geofetcher.const_limit_discard"></a>**`const_limit_discard`**
- <a id="geofetch.Geofetcher.const_limit_project"></a>**`const_limit_project`**
- <a id="geofetch.Geofetcher.disable_progressbar"></a>**`disable_progressbar`**
- <a id="geofetch.Geofetcher.discard_soft"></a>**`discard_soft`**
- <a id="geofetch.Geofetcher.file_pipeline_project"></a>**`file_pipeline_project`**
- <a id="geofetch.Geofetcher.file_pipeline_samples"></a>**`file_pipeline_samples`**
- <a id="geofetch.Geofetcher.filter_re"></a>**`filter_re`**
- <a id="geofetch.Geofetcher.filter_size"></a>**`filter_size`**
- <a id="geofetch.Geofetcher.fq_folder"></a>**`fq_folder`**
- <a id="geofetch.Geofetcher.geo_folder"></a>**`geo_folder`**
- <a id="geofetch.Geofetcher.just_metadata"></a>**`just_metadata`**
- <a id="geofetch.Geofetcher.just_object"></a>**`just_object`**
- <a id="geofetch.Geofetcher.max_prefetch_size"></a>**`max_prefetch_size`**
- <a id="geofetch.Geofetcher.max_soft_size"></a>**`max_soft_size`**
- <a id="geofetch.Geofetcher.metadata_expanded"></a>**`metadata_expanded`**
- <a id="geofetch.Geofetcher.metadata_root_full"></a>**`metadata_root_full`**
- <a id="geofetch.Geofetcher.picard_path"></a>**`picard_path`**
- <a id="geofetch.Geofetcher.processed"></a>**`processed`**
- <a id="geofetch.Geofetcher.project_name"></a>**`project_name`**
- <a id="geofetch.Geofetcher.refresh_metadata"></a>**`refresh_metadata`**
- <a id="geofetch.Geofetcher.skip"></a>**`skip`**
- <a id="geofetch.Geofetcher.split_experiments"></a>**`split_experiments`**
- <a id="geofetch.Geofetcher.sra_folder"></a>**`sra_folder`**
- <a id="geofetch.Geofetcher.supp_by"></a>**`supp_by`**
- <a id="geofetch.Geofetcher.use_key_subset"></a>**`use_key_subset`**

#### Methods

<a id="geofetch.Geofetcher.fetch_all"></a>

##### `fetch_all`

```python
def fetch_all(input: str, name: str | None = None) -> None | peppy.Project
```

Main function driver/workflow.

Searches, filters, downloads and saves data and metadata from GEO and SRA.

**Parameters:**

- `input` (*str*): GSE or input file with GSE accessions.
- `name` (*str | None*): Name of the project. (default: `None`)

**Returns:**

- *None | peppy.Project*: None or peppy Project.

<a id="geofetch.Geofetcher.fetch_processed_one"></a>

##### `fetch_processed_one`

```python
def fetch_processed_one(gse_file_content: list[str], gsm_file_content: list[str], gsm_filter_list: dict) -> tuple[list, list]
```

Fetch one processed GSE project and return its metadata.

**Parameters:**

- `gse_file_content` (*list[str]*): GSE soft file content.
- `gsm_file_content` (*list[str]*): GSM soft file content.
- `gsm_filter_list` (*dict*): List of GSM that have to be downloaded.

**Returns:**

- *tuple[list, list]*: Tuple of (meta_processed_samples, meta_processed_series).

<a id="geofetch.Geofetcher.get_projects"></a>

##### `get_projects`

```python
def get_projects(input: str, just_metadata: bool = True, discard_soft: bool = True) -> dict
```

Fetch projects from GEO|SRA and return peppy projects.

**Parameters:**

- `input` (*str*): GSE number, or path to file of GSE numbers.
- `just_metadata` (*bool*): Process only metadata. (default: `True`)
- `discard_soft` (*bool*): Clean run, without downloading soft files. (default: `True`)

**Returns:**

- *dict*: Peppy project or list of projects, if acc_anno is set.

### Finder Class

Class for searching and finding GSE accessions:

<a id="geofetch.Finder"></a>

### `Finder`

```python
class Finder(filters: str | None = None, retmax: int = RETMAX)
```

Class for finding GSE accessions in special period of time.

Initialize Finder with optional filters and result limit.

**Parameters:**

- `filters` (*str | None*): Filters that have to be added to the query.
  Filter Patterns can be found here:
  https://www.ncbi.nlm.nih.gov/books/NBK3837/#EntrezHelp.Using_the_Advanced_Search_Pag (default: `None`)
- `retmax` (*int*): Maximum number of retrieved accessions. (default: `RETMAX`)

#### Attributes

- <a id="geofetch.Finder.last_result"></a>**`last_result`**
- <a id="geofetch.Finder.query_customized_ending"></a>**`query_customized_ending`**
- <a id="geofetch.Finder.query_filter_str"></a>**`query_filter_str`**

#### Methods

<a id="geofetch.Finder.find_differences"></a>

##### `find_differences`

```python
@staticmethod
def find_differences(old_list: list[str], new_list: list[str]) -> list[str]
```

Compare 2 lists and search for elements that are not in old list.

**Parameters:**

- `old_list` (*list[str]*): Old list of elements.
- `new_list` (*list[str]*): New list of elements.

**Returns:**

- *list[str]*: List of elements that are not in old list but are in new_list.

<a id="geofetch.Finder.generate_file"></a>

##### `generate_file`

```python
def generate_file(file_path: str, gse_list: list[str] | None = None) -> None
```

Save the list of GSE accessions stored in this Finder object to a given file.

**Parameters:**

- `file_path` (*str*): Root to the file where gse accessions have to be saved.
- `gse_list` (*list[str] | None*): List of gse accessions. (default: `None`)

<a id="geofetch.Finder.get_gse_all"></a>

##### `get_gse_all`

```python
def get_gse_all() -> list[str]
```

Get list of all gse accessions available in GEO.

**Returns:**

- *list[str]*: List of gse accessions.

<a id="geofetch.Finder.get_gse_by_date"></a>

##### `get_gse_by_date`

```python
def get_gse_by_date(start_date: str, end_date: str | None = None) -> list[str]
```

Search gse accessions by providing start date and end date. By default, the last date is today.

**Parameters:**

- `start_date` (*str*): The oldest date of update (from YYYY/MM/DD to now) [input format: 'YYYY/MM/DD'].
- `end_date` (*str | None*): The nearest date of update (from __ to YYYY/MM/DD) [input format: 'YYYY/MM/DD']. (default: `None`)

**Returns:**

- *list[str]*: List of gse accessions.

<a id="geofetch.Finder.get_gse_by_day_count"></a>

##### `get_gse_by_day_count`

```python
def get_gse_by_day_count(n_days: int = 1) -> list[str]
```

Get list of gse accessions that were uploaded or updated in last X days.

**Parameters:**

- `n_days` (*int*): Number of days from now [e.g. 5]. (default: `1`)

**Returns:**

- *list[str]*: List of gse accessions.

<a id="geofetch.Finder.get_gse_id_by_query"></a>

##### `get_gse_id_by_query`

```python
def get_gse_id_by_query(url: str) -> list[str]
```

Run esearch (ncbi search tool) by specifying URL and retrieve gse list result.

**Parameters:**

- `url` (*str*): URL of the query.

**Returns:**

- *list[str]*: List of gse ids.

<a id="geofetch.Finder.get_gse_last_3_month"></a>

##### `get_gse_last_3_month`

```python
def get_gse_last_3_month() -> list[str]
```

Get list of gse accessions that were uploaded or updated in last 3 months.

**Returns:**

- *list[str]*: List of gse accessions.

<a id="geofetch.Finder.get_gse_last_week"></a>

##### `get_gse_last_week`

```python
def get_gse_last_week() -> list[str]
```

Get list of gse accessions that were uploaded or updated in last week.

**Returns:**

- *list[str]*: List of gse accessions.

<a id="geofetch.Finder.uid_to_gse"></a>

##### `uid_to_gse`

```python
@staticmethod
def uid_to_gse(uid: str) -> str
```

UID to GSE accession converter.

**Parameters:**

- `uid` (*str*): UID string (Unique Identifier Number in GEO).

**Returns:**

- *str*: GSE id string.