Following are four possible ways to organize content for different tasks.
| Strategy | Separation | Dependency Sharing | Complexity | |
|---|---|---|---|---|
| 1 | One Repo per Task | Total | Very Hard | 10/10 (Don't do it) |
| 2 | Repo per WP (Branches) | High | Hard (via Merges) | 7/10 (Could do it) |
| 3 | Repo per WP (Folders) | Logical | Easy (via Imports) | 3/10 (Recommended) |
| 4 | One Big Monorepo | Minimal | Effortless | 1/10 (Simplest) |
We are evaluating how to structure the project (5 Work Packages, multiple tasks involving code and documents). Below are the four primary strategies for managing dependencies and task flows:
Every single task (T1.1, T2.1, etc.) gets its own Git repository.
- Pros: Total isolation; teams can work on one task without seeing any other code.
- Cons: Extremely high overhead. Managing 7+ repositories for one project leads to "Dependency Hell." Updating a "feeder" task requires manual Pull Requests across multiple other repos.
- Best for: Massive organizations where teams are completely unrelated.
Use tools like DVC (Data Version Control) or Dagster/Prefect to manage the flow.
- Pros: Automation. If you change the code in T2.1, the system automatically knows which downstream tasks (T2.2, T2.4) need to be re-run. Great for Jupyter Notebook reproducibility.
- Cons: Higher learning curve. Requires writing configuration files (like dvc.yaml) to define the graph.
- Best for: Data-heavy projects or research where the "flow" is run frequently and must be audited.
We create one repository for each Work Package (e.g., Project-WP2). Inside, tasks are separated by folders, not branches.
- Pros: Good balance of isolation and usability. Tasks within a WP can easily share a shared_lib/ folder for common logic. Logic "feeds" into other tasks via simple Python imports.
- Cons: Requires a small amount of path configuration (or an "editable install") to allow folders to talk to each other.
- Best for: Projects with clear boundaries between Work Packages but high internal dependencies.
The entire project (All WPs and all Tasks) lives in one single repository with a structured folder system.
- Pros: Simplest management. One single commit history, one environment, and effortless code sharing via standard Python imports. High visibility for the whole team.
- Cons: The repository can become large over time; requires discipline in folder naming.
- Best for: Small to medium teams that need to move fast and keep logic in sync.