Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 7 additions & 55 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,33 +43,29 @@
<a href="https://github.com/hamstring-ndr/hamstring/actions/workflows/build_test_macos.yml">
<img src="https://img.shields.io/github/actions/workflow/status/hamstring-ndr/hamstring/build_test_macos.yml?branch=main&logo=apple&style=for-the-badge&label=macos" alt="MacOS WorkFlows" />
</a>
<a href="https://github.com/hamstring-ndr/hamstring/actions/workflows/build_test_windows.yml">
<img src="https://img.shields.io/github/actions/workflow/status/hamstring-ndr/hamstring/build_test_windows.yml?branch=main&logo=windows&style=for-the-badge&label=windows" alt="Windows WorkFlows" />
</a>
</td>
</tr>
</table>

## About the Project

![Pipeline overview](./assets/heidgaf_architecture.svg)
![Pipeline overview](./assets/hamstring_architecture.svg)

## Getting Started

#### Run **HAMSTRING** using Docker Compose:
```sh
HOST_IP=127.0.0.1 docker compose -f docker/docker-compose.yml --profile prod up
```
<p align="center">
<img src="https://raw.githubusercontent.com/hamstring-ndr/hamstring/main/assets/terminal_example.gif?raw=true" alt="Terminal example"/>
</p>

#### Use the dev profile for testing out changes in docker containers:
```sh
HOST_IP=127.0.0.1 docker compose -f docker/docker-compose.yml --profile dev up
```
<p align="center">
<img src="./assets/hamstring_terminal.gif" alt="Terminal example"/>
</p>


<p align="right">(<a href="#readme-top">back to top</a>)</p>



Expand All @@ -85,7 +81,7 @@ possibly infrastructure.

The section `pipeline.log_collection.collector.logline_format` has to be adjusted to reflect your specific input log
line format. Using our adjustable and flexible log line configuration, you can rename, reorder and fully configure each
field of a valid log line. Freely define timestamps, RegEx patterns, lists, and IP addresses. For example, your
field of a valid log line. You can freely define timestamps, RegEx patterns, lists, and IP addresses. For example, your
configuration might look as follows:

```yml
Expand Down Expand Up @@ -187,24 +183,11 @@ Have a look at the following pictures showing examples of how these dashboards m

</details>

<p align="right">(<a href="#readme-top">back to top</a>)</p>


## Models and Training

To train and test our and possibly your own models, we currently rely on the following datasets:

- [DGTA Benchmark](https://data.mendeley.com/datasets/2wzf9bz7xr/1)
- [DNS Tunneling Queries for Binary Classification](https://data.mendeley.com/datasets/mzn9hvdcxg/1)
- [UMUDGA - University of Murcia Domain Generation Algorithm Dataset](https://data.mendeley.com/datasets/y8ph45msv8/1)
- [DGArchive](https://dgarchive.caad.fkie.fraunhofer.de/)
- [DNS Exfiltration](https://data.mendeley.com/datasets/c4n7fckkz3/3)

We compute all features separately and only rely on the `domain` and `class` for binary classification.

### Inserting Data for Testing

For testing purposes, you can ingest PCAPs or tap on network interfaces using the zeek-based sensor in its `1.0.0` release. For more information on it, please refer to [the documentation](https://github.com/Hamstring-NDR/hamstring-zeek).
For testing purposes, you can ingest PCAPs or tap on network interfaces using the zeek-based sensor that is integrated into the docker-compose file. For more information on the sensor, please refer to [the documentation](https://github.com/Hamstring-NDR/hamstring-zeek).

### Training Your Own Models

Expand Down Expand Up @@ -260,33 +243,6 @@ The results will be saved per default to `./results`, if not configured otherwis
```
This will create a `rules.txt` file containing the innards of the model, explaining the rules it created.

<p align="right">(<a href="#readme-top">back to top</a>)</p>


### Data

> [!IMPORTANT]
> We support custom schemes.

Depending on your data and usecase, you can customize the data scheme to fit your needs.
The below configuration is part of the [main configuration file](./config.yaml) which is detailed in our [documentation](https://HAMSTRING.readthedocs.io/en/latest/usage.html#id2)

```yml
loglines:
fields:
- [ "timestamp", RegEx, '^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$' ]
- [ "status_code", ListItem, [ "NOERROR", "NXDOMAIN" ], [ "NXDOMAIN" ] ]
- [ "src_ip", IpAddress ]
- [ "dns_server_ip", IpAddress ]
- [ "domain_name", RegEx, '^(?=.{1,253}$)((?!-)[A-Za-z0-9-]{1,63}(?<!-)\.)+[A-Za-z]{2,63}$' ]
- [ "record_type", ListItem, [ "A", "AAAA" ] ]
- [ "response_ip", IpAddress ]
- [ "size", RegEx, '^\d+b$' ]
```



<p align="right">(<a href="#readme-top">back to top</a>)</p>

<!-- CONTRIBUTING -->
## Contributing
Expand All @@ -305,16 +261,12 @@ Don't forget to give the project a star! Thanks again!
</a>


<p align="right">(<a href="#readme-top">back to top</a>)</p>

<!-- LICENSE -->

## License

Distributed under the EUPL License. See `LICENSE.txt` for more information.

<p align="right">(<a href="#readme-top">back to top</a>)</p>


<!-- MARKDOWN LINKS & IMAGES -->
<!-- https://www.markdownguide.org/basic-syntax/#reference-style-links -->
Expand Down
File renamed without changes
Binary file added assets/hamstring_terminal.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed assets/terminal_example.gif
Binary file not shown.
2 changes: 1 addition & 1 deletion docker/docker-compose/prod/docker-compose.monitoring.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
services:
monitoring_agent:
image: ghcr.io/hamstring-ndr/hamstring-monitoring:v1.0.0
image: ghcr.io/hamstring-ndr/hamstring-monitoring:v2.0.0
restart: "unless-stopped"
volumes:
- ../../../config.yaml:/app/config.yaml
Expand Down
Binary file added docs/media/hamstring_architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions docs/media/hamstring_architecture.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 0 additions & 4 deletions docs/media/heidgaf_architecture.svg

This file was deleted.

Binary file removed docs/media/heidgaf_overview_detailed.drawio.png
Binary file not shown.
Binary file removed docs/media/monitoring_pipeline.png
Binary file not shown.
Binary file removed docs/media/pipeline_overview.png
Binary file not shown.
Loading