# Artifact Description

This artifact provides codes and results corresponding to the experimental setting used in the paper: **Limits of Clustering Models Based on Distance Similarity: A Sensitivity Study of SDOclust** by Sabina Khazari, Tanja Zseby, Félix Iglesias Vázquez.

Codes are designed to conduct analysis to evaluate SDOclust main hyperparameters sensitivity.

Aug 2026


## 1. Reported Results

Complete experimental results are provided in the *results.zip* file. Extract it as the [results] folder, which contains:

- [results/run20260430_213142] --> *outs* analysis.
- [results/run20260430_214742] --> *clus* analysis.
- [results/run20260430_220412] --> *inspread* analysis.
- [results/run20260430_222012] --> *outspread* analysis.
- [results/run20260430_223512] --> *dims* analysis.
- [results/run20260430_231155] --> *factorial* analysis.
- [results/run20260501_184618] --> *size* analysis.


## 2. Requirements

- Python 3.9.6  
- Linux or macOS (recommended)

All required Python packages are listed in `requirements_complete.txt`.

## 3. Setup

*You can alternatively user the Docker image (recommended). See section 8.*

Create and activate a virtual environment:

```bash
python3.9 -m venv venv
source venv/bin/activate
```

Install dependencies:

```bash
pip install -r requirements_complete.txt
```

## 4. Reproducing Experiments

Each experiment is configured via a JSON file and executed as:

```bash
python sensitivity_analysis.py --config configs/<config_name>.json
```

To run the full experimental suite:

```bash
bash run.sh
```

This executes all configurations in sequence:

```bash
python sensitivity_analysis.py --config configs/outs.json
python sensitivity_analysis.py --config configs/clus.json
python sensitivity_analysis.py --config configs/inspread.json
python sensitivity_analysis.py --config configs/outspread.json
python sensitivity_analysis.py --config configs/dims.json
python sensitivity_analysis.py --config configs/factorial.json
python sensitivity_analysis.py --config configs/size.json
```

### Configuration Files

Each JSON config specifies the analysis to run. Available `analysis_type` values:

- `size` — varies number of samples
- `dims` — varies number of features
- `clus` — varies number of clusters
- `outs` — varies outlier fraction
- `inspread` — varies inlier spread
- `outspread` — varies outlier spread
- `factorial` — joint sweep over clusters and inlier spread
- `all` — runs all of the above in sequence

Example config:

```json
{
  "analysis_type": "size",
  "i": 10,
  "o": "results",
  "seeds": "0,1,2"
}
```

## 5. Output

Results are stored in a timestamped subfolder inside the `results/` directory (e.g. `results/run20260501_143022/`).

Each run produces:

- `ALL_combinations_LONG.csv` — one row per (seed, dataset config, algorithm) with ARI, execution time, effective parameters used, and error info
- `args.json` — copy of the config used for that run

## 6. Reproducibility

- Python version is fixed (3.9.6)  
- Dependencies are fully specified in `requirements_complete.txt`  
- Random seeds are controlled via the `seeds` field in each config file
- A pre-built docker image (and the docker configuration files to build the image) to replicate the experimental environment.

## 6. Structure

- `sensitivity_analysis.py` — main experiment runner
- `configs/` — JSON configuration files for each analysis type
- `run.sh` — batch execution script
- `results/` — output folder with CSV results and run metadata

## 7. Notes

- The SDO algorithm grid is configured via the optional `sdo_grid` field in the JSON config.
- Results are written incrementally to CSV so experiments can be interrupted and resumed.
- ARI is computed on inliers only (outliers are excluded from clustering evaluation).

## 8. Docker

To facilitate reproducibility and reuse, we provide a pre-built Docker image that can either be loaded directly or rebuilt from the provided Docker configuration files.

Using the pre-built Docker image is the recommended option for reproducing the experiments, as it already contains the required Python environment and dependencies. In this case, there is no need to create or configure a Python virtual environment on the host system.

You need to have **Docker** and **Docker Compose** (or the Docker Compose plugin) installed on your system. The `docker/requirements.txt` file is exactly the same as `requirements_complete.txt` file.

> **Note:** Depending on your system configuration, Docker commands may require `sudo`. Alternatively, you can configure your user account to run Docker without `sudo`.

### Using the pre-built Docker image

The pre-built Docker image is the file: `docker-reproducible-env.tar`. Load it into your local Docker installation with:

```bash
docker load -i docker-reproducible-env.tar
```

Instead of loading the image, you can build it by using the Docker configuration files provided in the `docker` folder:

- `docker-compose.yml`
- `Dockerfile`
- `Makefile`
- `requirements.txt` (Python package dependencies)

The `Makefile` simplifies the management of the Docker container through the following commands:

- `build`: builds the Docker image using the available cache.
- `run`: run all the experiments.
- `shell`: opens a shell inside the running container.
- `clean`: stops and removes the Docker containers.
- `nuke`: **Warning:** removes all Docker images from the system.

The Docker container mounts the local artifact directory as a volume. Therefore, files generated by the container are available directly on the host system.