Skip to content

Commit 5ab3158

Browse files
vinod0mvinod0m
committed
refactor(agent): Rename SDLCFlexibleAgent to FlexibleAgent, update docs, workflows, and scripts
- Renamed main agent class from SDLCFlexibleAgent to FlexibleAgent in src/agents/deepagent.py; added alias for backward compatibility - Updated all documentation, examples, and CI workflows to prefer FlexibleAgent - Added deprecation note for SDLCFlexibleAgent in doc/deepagent.md - Harmonized copilot-instructions.md, AGENTS.md, and README.md for venv/test script workflow, coverage, lint/typecheck, and Makefile targets - Added Makefile targets for lint, lint-fix, typecheck, format, coverage, and unified ci - Added .pre-commit-config.yaml with ruff, mypy, and coverage hooks; updated requirements-dev.txt - Hardened provider normalization and made dotenv imports optional in scripts/examples - Verified all changes with full test suite (78/78 passing) BREAKING CHANGE: SDLCFlexibleAgent is deprecated; use FlexibleAgent for all new code. Alias remains for compatibility but will be removed in a future release.
1 parent 18c72f7 commit 5ab3158

22 files changed

+254
-96
lines changed

‎.pre-commit-config.yaml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
repos:
2+
- repo: https://github.com/astral-sh/ruff-pre-commit
3+
rev: v0.4.2
4+
hooks:
5+
- id: ruff
6+
args: ["--fix"]
7+
- id: ruff-format
8+
9+
- repo: https://github.com/pre-commit/mirrors-mypy
10+
rev: v1.9.0
11+
hooks:
12+
- id: mypy
13+
name: mypy (src)
14+
args:
15+
- "--ignore-missing-imports"
16+
- "--exclude=src/llm/router.py"
17+
files: ^src/.*\.py$
18+
19+
- repo: local
20+
hooks:
21+
- id: run-tests-with-coverage
22+
name: Run pytest with coverage (pre-push)
23+
entry: ./scripts/run-tests.sh --cov=src --cov-report=term-missing
24+
language: system
25+
pass_filenames: false
26+
stages: [push]

‎AGENTS.md‎

Lines changed: 37 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,33 @@
11
# Instructions for AI Agents
22

3-
This document provides instructions and guidelines for AI agents working with the SDLC_core repository.
3+
This document provides instructions and guidelines for AI agents working with the unstructuredDataHandler repository.
44

55
## Repository Overview
66

7-
SDLC_core is a Python-based Software Development Life Cycle core project that provides AI/ML capabilities for software development workflows. The repository contains modules for LLM clients, intelligent agents, memory management, prompt engineering, document retrieval, skill execution, and various utilities.
7+
unstructuredDataHandler is a Python-based Software Development Life Cycle core project that provides AI/ML capabilities for software development workflows. The repository contains modules for LLM clients, intelligent agents, memory management, prompt engineering, document retrieval, skill execution, and various utilities.
88

99
- **Primary Language**: Python 3.10-3.12
1010
- **Secondary Languages**: TypeScript (for Azure pipelines), Shell scripts
1111
- **Project Type**: AI/ML library and tooling for SDLC workflows
1212

1313
## Environment Setup
1414

15-
### 1. Install Dependencies
15+
### 1. Preferred: Isolated venv via script
1616

17-
**IMPORTANT**: The project's dependencies are split into multiple files. For development and testing, you should install the dependencies from `requirements-dev.txt`.
17+
Use the reproducible test script. It creates `.venv_ci` and pins pytest for reliable runs.
1818

1919
```bash
20+
./scripts/run-tests.sh
21+
```
22+
23+
### 2. Alternative: Local dev venv
24+
25+
Create and activate your own virtual environment, then install dev dependencies.
26+
27+
```bash
28+
python3 -m venv .venv
29+
source .venv/bin/activate
30+
pip install -U pip
2031
pip install -r requirements-dev.txt
2132
```
2233

@@ -38,16 +49,18 @@ PYTHONPATH=. python -m pytest
3849

3950
### Testing
4051

41-
The test infrastructure is set up. Use the following commands to run tests:
52+
Preferred (isolated venv):
4253

4354
```bash
44-
# Run all tests
45-
PYTHONPATH=. python -m pytest test/ -v
55+
./scripts/run-tests.sh # Full test run
56+
./scripts/run-tests.sh test/unit -k deepagent # Narrow selection
57+
```
4658

47-
# Run tests with coverage
48-
PYTHONPATH=. python -m pytest test/ --cov=src/ --cov-report=xml
59+
Alternative (local venv):
4960

50-
# Run specific test suites
61+
```bash
62+
PYTHONPATH=. python -m pytest test/ -v
63+
PYTHONPATH=. python -m pytest test/ --cov=src/ --cov-report=xml
5164
PYTHONPATH=. python -m pytest test/unit/ -v
5265
PYTHONPATH=. python -m pytest test/integration/ -v
5366
PYTHONPATH=. python -m pytest test/e2e/ -v
@@ -83,26 +96,27 @@ The core logic is in the `src/` directory, which is organized into the following
8396
- `src/utils/`: Logging, caching, rate limiting, tokens
8497

8598
Other important directories:
99+
86100
- `config/`: YAML configurations for models, prompts, logging
87101
- `data/`: Prompts, embeddings, dynamic content
88102
- `examples/`: Minimal scripts demonstrating key features
89103
- `test/`: Unit, integration, smoke, and e2e tests
90104

91105
## Key Development Rules
92106

93-
### ALWAYS:
107+
### ALWAYS
94108

95-
1. **Install dependencies** before making changes.
96-
2. **Set the `PYTHONPATH`** for all commands.
97-
3. **Run tests** (`PYTHONPATH=. python -m pytest test/ -v`) to validate the current state before making changes.
98-
4. **Configure the agent** by editing `config/model_config.yaml` before running it.
99-
5. **Ensure new Python modules** have proper `__init__.py` files.
100-
6. **Follow the branch naming convention**: `dev/<alias>/<feature>`.
101-
7. **Fill out the PR template** when submitting a pull request. The template is located at `.github/PULL_REQUEST_TEMPLATE.md`.
109+
1. **Install dependencies** before making changes.
110+
2. **Set the `PYTHONPATH`** for all commands.
111+
3. **Run tests** (`PYTHONPATH=. python -m pytest test/ -v`) to validate the current state before making changes.
112+
4. **Configure the agent** by editing `config/model_config.yaml` before running it.
113+
5. **Ensure new Python modules** have proper `__init__.py` files.
114+
6. **Follow the branch naming convention**: `dev/<alias>/<feature>`.
115+
7. **Fill out the PR template** when submitting a pull request. The template is located at `.github/PULL_REQUEST_TEMPLATE.md`.
102116

103-
### NEVER:
117+
### NEVER
104118

105-
- Run tests without setting `PYTHONPATH`.
106-
- Assume `requirements.txt` contains dependencies.
107-
- Create modules named "router" (conflicts with existing router.py files).
108-
- Modify Azure pipeline scripts (`build/azure-pipelines/`) without TypeScript knowledge.
119+
- Run tests without setting `PYTHONPATH`.
120+
- Assume `requirements.txt` contains dependencies.
121+
- Create modules named "router" (conflicts with existing router.py files).
122+
- Modify Azure pipeline scripts (`build/azure-pipelines/`) without TypeScript knowledge.

‎CONTRIBUTING.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -63,14 +63,14 @@ If you don't have any additional info/context to add but would like to indicate
6363

6464
If you're able & willing to help fix issues and/or implement features, we'd love your contribution!
6565

66-
The list of ["good first issue"](https://github.com/SoftwareDevLabs/SDLC_core/issues?q=is%3Aopen+is%3Aissue+label%3A%22Help+Wanted%22++label%3A%22good+first+issue%22+)s is another set of issues that might be easier for first-time contributors. Once you're feeling more comfortable in the codebase, feel free to just use the ["Help Wanted"](https://github.com/SoftwareDevLabs/SDLC_core/issues?q=is%3Aopen+is%3Aissue+label%3A%22Help+Wanted%22+) label, or just find any issue you're interested in and hop in!
66+
The list of ["good first issue"](https://github.com/SoftwareDevLabs/unstructuredDataHandler/issues?q=is%3Aopen+is%3Aissue+label%3A%22Help+Wanted%22++label%3A%22good+first+issue%22+)s is another set of issues that might be easier for first-time contributors. Once you're feeling more comfortable in the codebase, feel free to just use the ["Help Wanted"](https://github.com/SoftwareDevLabs/unstructuredDataHandler/issues?q=is%3Aopen+is%3Aissue+label%3A%22Help+Wanted%22+) label, or just find any issue you're interested in and hop in!
6767

6868
Generally, we categorize issues in the following way:
69-
* ["Bugs"](https://github.com/SoftwareDevLabs/SDLC_core/issues?q=is%3Aopen+is%3Aissue+label%3A%22Issue-Bug%22+) are parts of the SDLC_core that are not quite working the right way. There's code to already support some scenario, but it's not quite working right. Fixing these is generally a matter of debugging the broken functionality and fixing the wrong code.
70-
* ["Tasks"](https://github.com/SoftwareDevLabs/SDLC_core/issues?q=is%3Aopen+is%3Aissue+label%3A%22Issue-Task%22+) are usually new pieces of functionality that aren't yet implemented for the SDLC_core. These are usually smaller features, which we believe
69+
* ["Bugs"](https://github.com/SoftwareDevLabs/unstructuredDataHandler/issues?q=is%3Aopen+is%3Aissue+label%3A%22Issue-Bug%22+) are parts of the unstructuredDataHandler that are not quite working the right way. There's code to already support some scenario, but it's not quite working right. Fixing these is generally a matter of debugging the broken functionality and fixing the wrong code.
70+
* ["Tasks"](https://github.com/SoftwareDevLabs/unstructuredDataHandler/issues?q=is%3Aopen+is%3Aissue+label%3A%22Issue-Task%22+) are usually new pieces of functionality that aren't yet implemented for the unstructuredDataHandler. These are usually smaller features, which we believe
7171
- could be a single, atomic PR
7272
- Don't require much design consideration, or we've already written the spec for the larger feature they belong to.
73-
* ["Features"](https://github.com/SoftwareDevLabs/SDLC_core/issues?q=is%3Aopen+is%3Aissue+label%3A%22Issue-Feature%22+) are larger pieces of new functionality. These are usually things we believe would require larger discussion of how they should be implemented, or they'll require some complicated new settings. They might just be features that are composed of many individual tasks. Often times, with features, we like to have a spec written before development work is started, to make sure we're all on the same page (see below).
73+
* ["Features"](https://github.com/SoftwareDevLabs/unstructuredDataHandler/issues?q=is%3Aopen+is%3Aissue+label%3A%22Issue-Feature%22+) are larger pieces of new functionality. These are usually things we believe would require larger discussion of how they should be implemented, or they'll require some complicated new settings. They might just be features that are composed of many individual tasks. Often times, with features, we like to have a spec written before development work is started, to make sure we're all on the same page (see below).
7474

7575
Bugs and tasks are obviously the easiest to get started with, but don't feel afraid of features either! We've had some community members contribute some amazing "feature"-level work to our repos (albeit, with lots of discussion 😄).
7676

@@ -101,7 +101,7 @@ Team members will be happy to help review specs and guide them to completion.
101101

102102
### Help Wanted
103103

104-
Once the team has approved an issue/spec, development can proceed. If no developers are immediately available, the spec can be parked ready for a developer to get started. Parked specs' issues will be labeled "Help Wanted". To find a list of development opportunities waiting for developer involvement, visit the Issues and filter on [the Help-Wanted label](https://github.com/SoftwareDevLabs/SDLC_core/labels/Help%20Wanted).
104+
Once the team has approved an issue/spec, development can proceed. If no developers are immediately available, the spec can be parked ready for a developer to get started. Parked specs' issues will be labeled "Help Wanted". To find a list of development opportunities waiting for developer involvement, visit the Issues and filter on [the Help-Wanted label](https://github.com/SoftwareDevLabs/unstructuredDataHandler/labels/Help%20Wanted).
105105

106106
---
107107

@@ -130,7 +130,7 @@ Here are a few things you can do that will increase the likelihood of your pull
130130

131131
### Testing
132132

133-
Testing is a key component in the development workflow. This SDLC_core should use well defined testing methodology to ensure that SDLC_core and its key components are tested.
133+
Testing is a key component in the development workflow. This unstructuredDataHandler should use well defined testing methodology to ensure that unstructuredDataHandler and its key components are tested.
134134

135135
<!---TAEF (the Test Authoring and Execution Framework) as the main framework for testing.
136136

‎Makefile‎

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,26 @@
1-
.PHONY: test lint clean
1+
.PHONY: test lint lint-fix typecheck format ci clean coverage coverage-html
22

33
# Run tests using the reproducible script (creates .venv_ci)
44
test:
55
./scripts/run-tests.sh
66

7+
# Coverage reports (terminal summary)
8+
coverage:
9+
./scripts/run-tests.sh --cov=src --cov-report=term-missing
10+
11+
# Coverage reports (HTML)
12+
coverage-html:
13+
./scripts/run-tests.sh --cov=src --cov-report=html
14+
15+
# CI bundle: tests with coverage + ruff + mypy + pylint
16+
ci:
17+
./scripts/run-tests.sh --cov=src --cov-report=term-missing
18+
python3 -m pip install --upgrade pip
19+
pip install ruff mypy pylint
20+
python3 -m ruff check src/
21+
python3 -m mypy src/ --ignore-missing-imports --exclude="src/llm/router.py"
22+
python3 -m pylint src/ --exit-zero
23+
724
# Run lint and static checks
825
lint:
926
# Fast lint with ruff and type check with mypy
@@ -12,5 +29,23 @@ lint:
1229
python3 -m ruff check src/
1330
python3 -m mypy src/ --ignore-missing-imports
1431

32+
# Auto-fix lint issues with ruff
33+
lint-fix:
34+
python3 -m pip install --upgrade pip
35+
pip install ruff
36+
ruff check src/ --fix
37+
38+
# Type checking with mypy (with router exclusion)
39+
typecheck:
40+
python3 -m pip install --upgrade pip
41+
pip install mypy
42+
python3 -m mypy src/ --ignore-missing-imports --exclude="src/llm/router.py"
43+
44+
# Auto-format with ruff
45+
format:
46+
python3 -m pip install --upgrade pip
47+
pip install ruff
48+
ruff format src/
49+
1550
clean:
1651
rm -rf .venv_ci

‎README.md‎

Lines changed: 72 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11

22

3-
# Welcome to the SDLC_core Repo
3+
# Welcome to the unstructuredDataHandler Repo
44

55
<details>
66
<summary><strong>Table of Contents</strong></summary>
77

88
- [Installing and running Windows Terminal](#installing-and-running-windows-terminal)
9-
- [Module Roadmap](#SDLC_core-roadmap)
10-
- [SDLC_core Overview](#sdlc_core-overview)
9+
- [Module Roadmap](#unstructureddatahandler-roadmap)
10+
- [unstructuredDataHandler Overview](#unstructureddatahandler-overview)
1111
- [Resources](#resources)
1212
- [FAQ](#faq)
1313
- [Documentation](#documentation)
@@ -20,25 +20,25 @@
2020

2121
<br />
2222

23-
This repository contains the source code for the SDLC_core project, a Python-based framework for building AI-powered software development life cycle tools.
23+
This repository contains the source code for the unstructuredDataHandler project, a Python-based framework for building AI-powered software development life cycle tools.
2424

2525
Related repositories include:
2626

27-
* [SDLC_core Documentation](https://github.com/SoftwareDevLabs) (Placeholder)
27+
* [unstructuredDataHandler Documentation][docs-repo] (Placeholder)
2828

29-
## SDLC_core Roadmap
29+
## unstructuredDataHandler Roadmap
3030

31-
The plan for the SDLC_core [is described here](./doc/roadmap-20xx.md) and
31+
The plan for the unstructuredDataHandler [is described here](./doc/roadmap-20xx.md) and
3232
will be updated as the project proceeds.
3333

3434
## Installing and running Windows Terminal
3535

3636
> [!NOTE]
3737
> This section is a placeholder and may not be relevant to this project.
3838
39-
## SDLC_core Overview
39+
## unstructuredDataHandler Overview
4040

41-
SDLC_core is a Python-based Software Development Life Cycle core project that provides AI/ML capabilities for software development workflows. The repository contains modules for LLM clients, intelligent agents, memory management, prompt engineering, document retrieval, skill execution, and various utilities. It combines a Python core with TypeScript for Azure DevOps pipeline configurations.
41+
unstructuredDataHandler is a Python-based Software Development Life Cycle core project that provides AI/ML capabilities for software development workflows. The repository contains modules for LLM clients, intelligent agents, memory management, prompt engineering, document retrieval, skill execution, and various utilities. It combines a Python core with TypeScript for Azure DevOps pipeline configurations.
4242

4343
## Resources
4444

@@ -59,8 +59,8 @@ SDLC_core is a Python-based Software Development Life Cycle core project that pr
5959
## Documentation
6060

6161
All project documentation is located at [softwaremodule-docs](./doc/). If you would like
62-
to contribute to the documentation, please submit a pull request on the [SDLC_core
63-
Documentation](https://github.com/SoftwareDevLabs) repository.
62+
to contribute to the documentation, please submit a pull request on the [unstructuredDataHandler
63+
Documentation][docs-repo] repository.
6464

6565
---
6666

@@ -82,7 +82,7 @@ Documentation](https://github.com/SoftwareDevLabs) repository.
8282

8383
### Agents
8484

85-
The `agents` module provides the core components for creating AI agents. It includes a flexible `SDLCFlexibleAgent` that can be configured to use different LLM providers (like OpenAI, Gemini, and Ollama) and a set of tools. The module is designed to be extensible, allowing for the creation of custom agents with specialized skills. Key components include a planner and an executor (currently placeholders for future development) and a `MockAgent` for testing and CI.
85+
The `agents` module provides the core components for creating AI agents. It includes a flexible `FlexibleAgent` (formerly `SDLCFlexibleAgent`) that can be configured to use different LLM providers (like OpenAI, Gemini, and Ollama) and a set of tools. The module is designed to be extensible, allowing for the creation of custom agents with specialized skills. Key components include a planner and an executor (currently placeholders for future development) and a `MockAgent` for testing and CI.
8686

8787
### Parsers
8888

@@ -159,8 +159,48 @@ You can also use the Makefile targets:
159159
```bash
160160
make test
161161
make lint
162+
make typecheck
163+
make format
164+
make coverage # terminal coverage summary
165+
make coverage-html # generate HTML report in ./htmlcov/
166+
make ci # tests with coverage + ruff + mypy + pylint
162167
```
163168

169+
### How to test locally (two options)
170+
171+
- Preferred (isolated venv): Use `./scripts/run-tests.sh`. It creates `.venv_ci`, pins pytest, and runs with `PYTHONPATH` set correctly.
172+
- Alternative (your own venv):
173+
1. `python3 -m venv .venv`
174+
2. `source .venv/bin/activate`
175+
3. `pip install -U pip`
176+
4. `pip install -r requirements-dev.txt`
177+
5. `PYTHONPATH=. python -m pytest test/ -v`
178+
179+
### Optional: run with coverage
180+
181+
- Isolated venv script (add flags after the script path):
182+
- `./scripts/run-tests.sh --cov=src --cov-report=term-missing`
183+
- Local venv (after installing `requirements-dev.txt`):
184+
- `PYTHONPATH=. python -m pytest test/ --cov=src --cov-report=term-missing`
185+
186+
Makefile shortcuts:
187+
188+
- `make coverage` — terminal summary
189+
- `make coverage-html` — generates an HTML report in `./htmlcov/`
190+
191+
### Quick lint and type checks
192+
193+
- Makefile shortcut:
194+
- `make lint`
195+
- `make lint-fix` # ruff check with autofix
196+
- `make typecheck` # mypy with router exclusion
197+
- `make format` # ruff formatter
198+
- Manual (useful in CI or local venv):
199+
- `python -m pylint src/ --exit-zero`
200+
- `python -m mypy src/ --ignore-missing-imports --exclude="src/llm/router.py"`
201+
202+
Note: The mypy exclusion for `src/llm/router.py` avoids a duplicate module conflict with `src/fallback/router.py` during type analysis.
203+
164204
---
165205

166206
## Contributing
@@ -170,6 +210,20 @@ We are excited to work with the community to build and enhance this project.
170210
***BEFORE you start work on a feature/fix***, please read & follow our [Contributor's Guide](./CONTRIBUTING.md) to
171211
help avoid any wasted or duplicate effort.
172212
213+
### Developer setup: pre-commit hooks (optional but recommended)
214+
215+
To keep code quality consistent, we provide pre-commit hooks for ruff (lint+format) and mypy; and a pre-push hook that runs tests with coverage.
216+
217+
1. Install dev deps (once): `pip install -r requirements-dev.txt`
218+
2. Install hooks (once): `pre-commit install --install-hooks`
219+
3. Optional: enable pre-push test runner: `pre-commit install --hook-type pre-push`
220+
221+
Hooks configured in `.pre-commit-config.yaml`:
222+
223+
- ruff (with autofix) and ruff-format
224+
- mypy with the router exclusion
225+
- pre-push: `./scripts/run-tests.sh --cov=src --cov-report=term-missing`
226+
173227
## Communicating with the Team
174228
175229
The easiest way to communicate with the team is via GitHub issues.
@@ -189,18 +243,18 @@ Please review these brief docs below about our coding practices.
189243
This is a work in progress as we learn what we'll need to provide people in
190244
order to be effective contributors to our project.
191245
192-
- [Coding Style](./doc/STYLE.md)
193-
- [Code Organization](./doc/ORGANIZATION.md)
194-
- [Exceptions in our legacy codebase](./doc/EXCEPTIONS.md)
246+
- [Coding Style](./doc/STYLE.md)
247+
- [Code Organization](./doc/ORGANIZATION.md)
248+
- [Exceptions in our legacy codebase](./doc/EXCEPTIONS.md)
195249
196250
---
197251
198252
## Code of Conduct
199253
200254
This project has adopted the [Code of Conduct][conduct-code]. For more information see the [Code of Conduct][conduct-code] or contact [info@softwaredevlabs.com][conduct-email] with any additional questions or comments.
201255
202-
[conduct-code](./CODE_OF_CONDUCT.md)
203-
204-
[conduct-email]: mailto:info@softwaredevlabs.com
205256
[conduct-code]: ./CODE_OF_CONDUCT.md
206257
[conduct-email]: mailto:info@softwaredevlabs.com
258+
[docs-repo]: https://github.com/SoftwareDevLabs
259+
<!-- TODO: update [docs-repo] once the dedicated docs repository is created,
260+
e.g. https://github.com/SoftwareDevLabs/unstructuredDataHandler-docs -->

0 commit comments

Comments
 (0)