Display row number in each task and add seed command

* Add a seed command

* Add row number in templates

* Correctly set docker-compose's env

* No need for directory listing

* Clean unused code

* Use only moonsheep's production mode

* Show for errors in html

* Add support for row number in models and forms

* Adapt tasks to moonsheep's production mode

* Wait for a longer period of time

* Seed all the time

* Update development instructions

* Remove debug leftovers

Author: vtemian

README.md

@@ -1,4 +1,4 @@
-# Catalog politic  - Declaratii de avere [![Datree](https://s3.amazonaws.com/catalog.static.datree.io/datree-badge-20px.svg)](https://datree.io/?src=badge) [![GitHub contributors](https://img.shields.io/github/contributors/code4romania/catpol-declaratii.svg)](https://github.com/code4romania/catpol-declaratii/graphs/contributors) [![GitHub last commit](https://img.shields.io/github/last-commit/code4romania/catpol-declaratii.svg)](https://github.com/code4romania/catpol-declaratii/commits/master) [![License: MPL 2.0](https://img.shields.io/badge/license-MPL%202.0-brightgreen.svg)](https://opensource.org/licenses/MPL-2.0)
+# Catalog politic  - Declaratii de avere [![Datree](https://s3.amazonaws.com/catalog.static.datree.io/datree-badge-20px.svg)](https://datree.io/?src=badge) [![GitHub contributors](https://img.shields.io/github/contributors/code4romania/asset-declarations.svg)](https://github.com/code4romania/asset-declarations/graphs/contributors) [![GitHub last commit](https://img.shields.io/github/last-commit/code4romania/asset-declarations.svg)](https://github.com/code4romania/asset-declarations/commits/master) [![License: MPL 2.0](https://img.shields.io/badge/license-MPL%202.0-brightgreen.svg)](https://opensource.org/licenses/MPL-2.0)
 
 <!-- Please don't remove this: Grab your social icons from https://github.com/carlsednaoui/gitsocial -->
 
@@ -23,7 +23,7 @@
 * MAKING PUBLIC INFORMATION TRULY PUBLIC
 * data from asset declarations, including a net worth estimation
 
-**Important!** This project is currently out of sync with the latest version of [Moonsheep](https://github.com/themoonsheep/moonsheep), a library that it is dependent on. In order for development to continue, we must first fix [this critical issue](https://github.com/code4romania/catpol-declaratii/issues/191).
+**Important!** This project is currently out of sync with the latest version of [Moonsheep](https://github.com/themoonsheep/moonsheep), a library that it is dependent on. In order for development to continue, we must first fix [this critical issue](https://github.com/code4romania/asset-declarations/issues/191).
 
 Currently, in Romania, public information on elected officials is spread on a multitude of media, in a multitude of formats and requires a
 priori knowledge of the sources where data resides, making it hard, if not impossible for a regular citizen to make sense of the data.
@@ -87,34 +87,21 @@ Political Catalogue - Asset Declaration is a web application.
 Installation process
 * Fork this repo
 * Clone your fork
-* Open the directory where you have cloned the repo (`cd catpol-declaratii`)
+* Open the directory where you have cloned the repo (`cd asset-declarations`)
 * Optionally, you can create a virtual environment named "venv": `python3 -m venv venv` and then activate it: `source venv/bin/activate`
 * `pip install -r requirements-dev.txt` 
 * `export DJANGO_SETTINGS_MODULE=project_template.settings.dev`
 * `python manage.py migrate --run-syncdb`
+* `python manage.py seed`
 * `python manage.py runserver`
 
-Using Dockerfile:
-* Install docker
+Using docker-compose:
+* Install docker and cdocker-compose
 * Fork this repo
 * Clone your fork
-* Open the directory where you have cloned the repo (`cd catpol-declaratii`)
-* Run the following command to create a Docker image for the project `docker build -t catpol`
-* Run the following command to run the Docker image `docker run -p 8000 catpol`
-* Show the container id running the `catpol` image `docker ps`
-* Inspect the container to get the host port `docker inspect <container_id>`, you should see something like:
-```json
-            "Ports": {
-                "8000/tcp": [
-                    {
-                        "HostIp": "0.0.0.0",
-                        "HostPort": "32769"
-                    }
-                ]
-            },
- ```
- * Connect to specified port on localhost and enjoy the solution
-
+* Open the directory where you have cloned the repo (`cd asset-declarations`)
+* Run the following command to start the development server `docker-compose up`
+* You can interact with the container's environment using `docker-compose exec web bash`
 
 ## Contributing 
 

docker-compose.yml

@@ -2,7 +2,7 @@ version: '3'
 services:
   web:
       build: .
-      command: ls -la /opt/catpol && dockerize --wait tcp://db:3306
+      command: dockerize -wait tcp://db:5432 -timeout 90s
       ports:
         - "8000:8000"
       volumes:
@@ -12,6 +12,8 @@ services:
     image: "postgres:12"
     ports:
       - "5432:5432"
+    env_file:
+      - .env
     volumes:
       - database-data:/var/lib/postgresql/data/
 

docker-entrypoint

@@ -5,5 +5,8 @@ PORT=${PORT:-"8000"}
 echo "Migrating databse"
 python3 manage.py migrate --run-syncdb
 
+echo "Seed"
+python3 manage.py seed
+
 echo "Start web server on $PORT"
 python3 manage.py runserver "0.0.0.0:$PORT"

project_template/forms.py

@@ -76,7 +76,7 @@ def __init__(self, *args, **kwargs):
     class Meta:
         model = models.OwnedLandTableEntry
         # Exclude the Model's table and coowner fields because they will be handled separately by the Task
-        exclude = ["table", "coowner"]
+        exclude = ["table", "row_number", "coowner"]
 
 
 class TranscribeOwnedBuildingsTable(forms.Form):
@@ -101,7 +101,7 @@ def __init__(self, *args, **kwargs):
     class Meta:
         model = models.OwnedBuildingsTableEntry
         # Exclude the Model's table and coowner fields because they will be handled separately by the Task
-        exclude = ["table", "coowner"]
+        exclude = ["table", "row_number", "coowner"]
         # TODO: The 'address' model field doesn't seem to be used by the old form. Is it an overlook?
         exclude += ["address"]
 
@@ -129,6 +129,7 @@ class Meta:
         # Exclude the Model's table field because they will be handled separately by the Task
         exclude = [
             "table",
+            "row_number",
         ]
 
 
@@ -154,6 +155,7 @@ class Meta:
         # Exclude the Model's table field because they will be handled separately by the Task
         exclude = [
             "table",
+            "row_number",
         ]
 
 
@@ -182,7 +184,7 @@ def __init__(self, *args, **kwargs):
     class Meta:
         model = models.OwnedExtraValuableTableEntry
         # Exclude the Model's table and receiver_of_goods fields because they will be handled separately by the Task
-        exclude = ["table", "receiver_of_goods"]
+        exclude = ["table", "row_number", "receiver_of_goods"]
         # TODO: The 'address' model field doesn't seem to be used by the old form. Is it an overlook?
         exclude += ["address"]
 
@@ -201,6 +203,7 @@ class Meta:
         model = models.OwnedBankAccountsTableEntry
         exclude = [
             "table",
+            "row_number",
         ]
 
     def __init__(self, *args, **kwargs):
@@ -226,7 +229,7 @@ class TranscribeOwnedInvestmentsOver5KRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedInvestmentsOver5KTableEntry
         # Exclude the Model's table and loan_beneficiary fields because they will be handled separately by the Task
-        exclude = ["table", "loan_beneficiary"]
+        exclude = ["table", "row_number", "loan_beneficiary"]
 
 
 class TranscribeOwnedDebtsTable(forms.Form):
@@ -244,7 +247,7 @@ class TranscribeOwnedDebtsRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedDebtsTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]
 
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
@@ -271,7 +274,7 @@ class TranscribeOwnedGoodsOrServicesRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedGoodsOrServicesTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]
 
 
 class TranscribeOwnedIncomeFromSalariesTable(forms.Form):
@@ -288,7 +291,7 @@ class TranscribeOwnedIncomeFromSalariesRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedIncomeFromSalariesTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]
 
 
 class TranscribeIndependentActivitiesTable(forms.Form):
@@ -307,7 +310,7 @@ class TranscribeIndependentActivitiesRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedIncomeFromIndependentActivitiesTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]
 
 
 class TranscribeOwnedIncomeFromDeferredUseOfGoodsTable(forms.Form):
@@ -324,7 +327,7 @@ class TranscribeOwnedIncomeFromDeferredUseOfGoodsRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedIncomeFromDeferredUseOfGoodsTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]
 
 
 class TranscribeOwnedIncomeFromInvestmentsTable(forms.Form):
@@ -343,7 +346,7 @@ class TranscribeOwnedIncomeFromInvestmentsRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedIncomeFromInvestmentsTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]
 
 
 class TranscribeOwnedIncomeFromPensionsTable(forms.Form):
@@ -360,7 +363,7 @@ class TranscribeOwnedIncomeFromPensionsRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedIncomeFromPensionsTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]
 
 
 class TranscribeOwnedIncomeFromAgriculturalActivitiesTable(forms.Form):
@@ -377,7 +380,7 @@ class TranscribeOwnedIncomeFromAgriculturalActivitiesRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedIncomeFromAgriculturalActivitiesTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]
 
 
 class TranscribeOwnedIncomeFromGamblingTable(forms.Form):
@@ -394,7 +397,7 @@ class TranscribeOwnedIncomeFromGamblingRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedIncomeFromGamblingTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]
 
 
 class TranscribeOwnedIncomeFromOtherSourcesTable(forms.Form):
@@ -411,4 +414,4 @@ class TranscribeOwnedIncomeFromOtherSourcesRowEntry(PartialModelForm):
     class Meta:
         model = models.OwnedIncomeFromOtherSourcesTableEntry
         # Exclude the Model's table and person fields because they will be handled separately by the Task
-        exclude = ["table", "person"]
+        exclude = ["table", "row_number", "person"]

project_template/management/__init__.py

@@ -0,0 +1,21 @@
+from django.core.management.base import BaseCommand
+
+from moonsheep.models import Task
+from moonsheep.settings import MOONSHEEP
+
+from project_template.models import Declaration
+
+
+class Command(BaseCommand):
+    def handle(self, *args, **kwargs):
+        for url in [
+            "http://cdep.ro/declaratii/deputati/2016/avere/002a.pdf",
+            "http://cdep.ro/declaratii/deputati/2016/avere/003c.pdf",
+            "http://cdep.ro/declaratii/deputati/2016/avere/004e.pdf",
+            "http://cdep.ro/declaratii/deputati/2016/avere/005c.pdf",
+            "http://cdep.ro/declaratii/deputati/2016/avere/006d.pdf",
+        ]:
+            declaration, _ = Declaration.objects.get_or_create(url=url)
+
+            for task_type in MOONSHEEP["DOCUMENT_INITIAL_TASKS"]:
+                Task.objects.get_or_create(type=task_type, doc_id=declaration.id, params={"url": url,})