Skip to content

[wip] add: code submission endpoint #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft

[wip] add: code submission endpoint #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5d14f73
add: code submission endpoint
matheusfrancisco Apr 7, 2024
bcd7cef
fix lint
matheusfrancisco Apr 7, 2024
6034998
add: adpater, and controller to submit code execution
matheusfrancisco Apr 8, 2024
5a07925
fix: integration test
matheusfrancisco Apr 9, 2024
b7cdb3e
unused binding
matheusfrancisco Apr 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 16 additions & 10 deletions deps.edn
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,25 @@
{:paths ["src" "resources"]
:deps {org.clojure/clojure {:mvn/version "1.11.2"}
:deps {org.clojure/clojure {:mvn/version "1.11.2"}
com.github.parenthesin/components {:mvn/version "0.2.5"
:exclusions [prismatic/schema]}
com.github.seancorfield/honeysql {:mvn/version "2.6.1126"}
metosin/malli {:mvn/version "0.15.0"}
metosin/reitit-swagger {:mvn/version "0.6.0"}
org.postgresql/postgresql {:mvn/version "42.7.3"}}
org.clj-commons/digest {:mvn/version "1.4.100"}
com.github.seancorfield/honeysql {:mvn/version "2.6.1126"}
metosin/malli {:mvn/version "0.15.0"}
metosin/reitit-swagger {:mvn/version "0.6.0"}
org.postgresql/postgresql {:mvn/version "42.7.3"}}
:aliases
{:dev {:extra-paths ["test" "dev"]}
{:dev {:extra-paths ["test" "dev"]
:extra-deps {org.clojars.bigsy/pg-embedded-clj {:mvn/version "1.0.1"}
lambdaisland/kaocha {:mvn/version "1.88.1376"}
lambdaisland/kaocha-cloverage {:mvn/version "1.1.89"}
nubank/matcher-combinators {:mvn/version "3.9.1"}
nubank/state-flow {:mvn/version "5.15.0"}}}
:test {:extra-paths ["test"]
:extra-deps {org.clojars.bigsy/pg-embedded-clj {:mvn/version "1.0.1"}
lambdaisland/kaocha {:mvn/version "1.88.1376"}
lambdaisland/kaocha-cloverage {:mvn/version "1.1.89"}
nubank/matcher-combinators {:mvn/version "3.9.1"}
nubank/state-flow {:mvn/version "5.15.0"}}
lambdaisland/kaocha {:mvn/version "1.88.1376"}
lambdaisland/kaocha-cloverage {:mvn/version "1.1.89"}
nubank/matcher-combinators {:mvn/version "3.9.1"}
nubank/state-flow {:mvn/version "5.15.0"}}
:main-opts ["-m" "kaocha.runner"]}

:clojure-lsp {:replace-deps {com.github.clojure-lsp/clojure-lsp-standalone {:mvn/version "2024.03.13-13.11.00"}}
Expand Down
6 changes: 3 additions & 3 deletions dev/user.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
(ns user
(:require [malli.dev :as dev]
[malli.dev.pretty :as pretty]
[microservice-boilerplate.server]))
(:require [codes.clj.contest.submission-runner.server]
[malli.dev :as dev]
[malli.dev.pretty :as pretty]))

(defn start
[]
Expand Down
12 changes: 12 additions & 0 deletions src/codes/clj/contest/submission_runner/adapters/submission.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
(ns codes.clj.contest.submission-runner.adapters.submission
(:require [codes.clj.contest.submission-runner.wire.db.submission :as wire.db.submission]
[codes.clj.contest.submission-runner.wire.in.submission :as wire.in.submission]))

(defn wire->internal
{:malli/schema [:=> [:cat wire.in.submission/Submission] wire.db.submission/Submission]}
[{:keys [id code code-hash language test-cases]}]
{:submission/id id
:submission/code code
:submission/code_hash code-hash
:submission/language language
:submission/test_cases test-cases})
5 changes: 5 additions & 0 deletions src/codes/clj/contest/submission_runner/controllers/submission.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
(ns codes.clj.contest.submission-runner.controllers.submission)

(defn submit-code-execution!
[_submission]
(random-uuid))
Copy link
Member

@rafaeldelboni rafaeldelboni Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are generating this just to insert into the database, you can delegate the UUID generation to postgres.

Copy link
Member Author

@matheusfrancisco matheusfrancisco Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still not 100% clear what we are going to save on postgres
id, code, code-hash, language and test-cases as string?

Copy link
Member

@rafaeldelboni rafaeldelboni Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question, I think I would store information to be possible to trace back the author, contest and the submission.

I would have an table to the contests (where the base challenge, and test inputs (and demo test inputs) would be stored, another for the authors, another for the submission where you would have the challenge id, author id, and the submission itself, to not store to much data in the database I would keep only the latest submission for each challenge and author, so if I submit and answer for a challenge we update the submission table by author-id and challenge-id with the new code, code-hash and updated date

After I would have a table with the execution results where you could store submission-id, code-hash and the result.

We can whiteboard this later if you want so.

Copy link
Member Author

@matheusfrancisco matheusfrancisco Apr 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can whiteboard this later if you want so.

I would love to whiteboard this

12 changes: 12 additions & 0 deletions src/codes/clj/contest/submission_runner/ports/http_in/submission.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
(ns codes.clj.contest.submission-runner.ports.http-in.submission
(:require [codes.clj.contest.submission-runner.adapters.submission :as adapters.submission]
[codes.clj.contest.submission-runner.controllers.submission :as controllers.submission]))

(defn submit-code-execution!
[{{submission :body} :parameters
_components :components}]
(let [id (-> submission
(adapters.submission/wire->internal)
(controllers.submission/submit-code-execution!))]
{:status 201 :body {:id id}}))

22 changes: 19 additions & 3 deletions src/codes/clj/contest/submission_runner/routes.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
(ns codes.clj.contest.submission-runner.routes
(:require [codes.clj.contest.submission-runner.ports.http-in :as ports.http-in]
[codes.clj.contest.submission-runner.ports.http-in.submission :as ports.http-in.submission]
[codes.clj.contest.submission-runner.schemas.wire-in :as schemas.wire-in]
[codes.clj.contest.submission-runner.wire.in.submission :as wire.in.submission]
[codes.clj.contest.submission-runner.wire.out.submission :as wire.out.submission]
[reitit.swagger :as swagger]))

(def routes
[["/swagger.json"
{:get {:no-doc true
:swagger {:info {:title "btc-wallet"
:description "small sample using the microservice-boilerplate"}}
:swagger {:info {:title "Submission Runner"
:description "Submission Runner API"}}
:handler (swagger/create-swagger-handler)}}]

["/wallet"
Expand All @@ -32,4 +35,17 @@
:responses {201 {:body schemas.wire-in/WalletEntry}
400 {:body :string}
500 {:body :string}}
:handler ports.http-in/do-withdrawal!}}]]])
:handler ports.http-in/do-withdrawal!}}]]

;; submitions routes
["/code"
{:swagger {:tags ["Code Runner"]}}
["/submission"
Copy link
Member

@rafaeldelboni rafaeldelboni Apr 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think even we going to receive this by another backend we need to ensure a bearer token is passed, since this will be accessed by only one service, this could be stored in the configuration files

{:post {:summary "Submit to execution your code"
:parameters {:body wire.in.submission/Submission}
:responses {201 {:body wire.out.submission/SubmissionResult}
400 {:body :string}
500 {:body :string}}
:handler ports.http-in.submission/submit-code-execution!}}]]])


13 changes: 13 additions & 0 deletions src/codes/clj/contest/submission_runner/wire/db/submission.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
(ns codes.clj.contest.submission-runner.wire.db.submission)

(def Submission
[:map
[:submission/id string?]
Copy link
Member

@rafaeldelboni rafaeldelboni Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UUID?

[:submission/code string?]
[:submission/code_hash string?]
[:submission/language [:enum :clojure]]
[:submission/test_cases
[:map-of :keyword
[:map
[:input :any]
[:output :any]]]]])
30 changes: 30 additions & 0 deletions src/codes/clj/contest/submission_runner/wire/in/submission.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
(ns codes.clj.contest.submission-runner.wire.in.submission)

(def Submission
[:map
[:id string?]
[:code string?]
[:code-hash string?]
[:language [:enum :clojure]]
[:test-cases [:map-of :keyword
[:map
[:input :any]
[:output :any]]]]])

(comment
(require '[malli.generator :as m])
(require '[malli.core :as m.core])
(m/generate Submission)
(m.core/validate
Submission
{:code "(ns runner
(:require [clojure.test :refer [use-fixtures]])
(defn my-sum [a b] (+ a b)))"
:language :clojure
:test-cases {:case-1 {:input "(my-sum 1 2)"
:output 3}
:case-2 {:input "(my-sum 2 3)"
:output 5}}})
Comment on lines +20 to +27
Copy link
Member Author

@matheusfrancisco matheusfrancisco Apr 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rafaeldelboni what do you think of this input?

Copy link
Member Author

@matheusfrancisco matheusfrancisco Apr 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this way I will have to merge it, so I think is better to receive it already merged,

Copy link
Member

@rafaeldelboni rafaeldelboni Apr 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who is going to send this payload the frontend?

Copy link
Member

@rafaeldelboni rafaeldelboni Apr 7, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is use this code + test cases to build the code that will run in the pod right?

Copy link
Member Author

@matheusfrancisco matheusfrancisco Apr 7, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who is going to send this payload the frontend?

Hm, 🤔 idk why I was thinking of having two backends.

Copy link
Member Author

@matheusfrancisco matheusfrancisco Apr 7, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is use this code + test cases to build the code that will run in the pod right?

Yep, the idea was to merge it in some way and send to the pod

Copy link
Member

@rafaeldelboni rafaeldelboni Apr 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:language could be an enum WDYT?

Copy link
Member Author

@matheusfrancisco matheusfrancisco Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice 👍🏽


;
)
5 changes: 5 additions & 0 deletions src/codes/clj/contest/submission_runner/wire/out/submission.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
(ns codes.clj.contest.submission-runner.wire.out.submission)

(def SubmissionResult
[:map
[:id uuid?]])
2 changes: 1 addition & 1 deletion ...clj/contest/submission_runner/db_test.clj → .../contest/submission_runner/db/db_test.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
(ns integration.codes.clj.contest.submission-runner.db-test
(ns integration.codes.clj.contest.submission-runner.db.db-test
(:require [clojure.test :refer [use-fixtures]]
[codes.clj.contest.submission-runner.db :as db]
[com.stuartsierra.component :as component]
Expand Down
40 changes: 40 additions & 0 deletions test/integration/codes/clj/contest/submission_runner/submission_runner_test.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
(ns integration.codes.clj.contest.submission-runner.submission-runner-test
(:require [clj-commons.digest :as digest]
[clojure.test :refer [use-fixtures]]
[integration.codes.clj.contest.submission-runner.util :as util]
[matcher-combinators.matchers :as matchers]
[parenthesin.helpers.malli :as helpers.malli]
[parenthesin.helpers.state-flow.server.pedestal :as state-flow.server]
[state-flow.api :refer [defflow]]
[state-flow.assertions.matcher-combinators :refer [match?]]
[state-flow.core :as state-flow :refer [flow]]))

(use-fixtures :once helpers.malli/with-intrumentation)

(defflow
flow-integration-wallet-test
{:init util/start-system!
:cleanup util/stop-system!
:fail-fast? true}

(flow "should receive an submission"
[:let [id (random-uuid)
code "(ns runner
(:require [clojure.string :as str]))
(defn my-sum [a b] (+ a b))"
submission-input {:id id
:code code
:code-hash (digest/md5 code)
:language :clojure
:test-cases {:case-1 {:input "(my-sum 1 2)"
:output 3}
:case-2 {:input "(my-sum 2 3)"
:output 5}}}]
return (state-flow.server/request! {:method :post
:uri "/code/submission"
:body submission-input})]

(match? (matchers/embeds {:status 201
:body {:id string?}})
return)))

34 changes: 27 additions & 7 deletions test/integration/codes/clj/contest/submission_runner/util.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,38 @@
(ns integration.codes.clj.contest.submission-runner.util
(:require [com.stuartsierra.component :as component]
(:require [codes.clj.contest.submission-runner.routes :as routes]
[com.stuartsierra.component :as component]
[parenthesin.components.config.aero :as components.config]
[parenthesin.components.db.jdbc-hikari :as components.database]
[parenthesin.components.http.clj-http :as components.http]
[parenthesin.components.router.reitit-malli :as components.router]
[parenthesin.components.server.reitit-pedestal-jetty :as components.webserver]
[parenthesin.helpers.logs :as logs]
[parenthesin.helpers.migrations :as migrations]
[pg-embedded-clj.core :as pg-emb]))

(defn- create-and-start-components! []
(component/start-system
(component/system-map
:config (components.config/new-config)
:http (components.http/new-http-mock {})
:router (components.router/new-router routes/routes)
:database (component/using (components.database/new-database)
[:config])
:webserver (component/using (components.webserver/new-webserver)
[:config :http :router :database]))))

(defn start-system!
[system-start-fn]
(fn []
(logs/setup :info :auto)
(pg-emb/init-pg)
(migrations/migrate (migrations/configuration-with-db))
(system-start-fn)))
([]
((start-system! create-and-start-components!)))
([system-start-fn]
(fn []
(logs/setup :info :auto)
(pg-emb/init-pg)
(migrations/migrate (migrations/configuration-with-db))
(system-start-fn))))

(defn stop-system!
[system]
(component/stop-system system)
(pg-emb/halt-pg!))

20 changes: 1 addition & 19 deletions test/integration/codes/clj/contest/submission_runner/wallet_test.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,7 @@
(ns integration.codes.clj.contest.submission-runner.wallet-test
(:require [clojure.test :refer [use-fixtures]]
[codes.clj.contest.submission-runner.routes :as routes]
[com.stuartsierra.component :as component]
[integration.codes.clj.contest.submission-runner.util :as util]
[matcher-combinators.matchers :as matchers]
[parenthesin.components.config.aero :as components.config]
[parenthesin.components.db.jdbc-hikari :as components.database]
[parenthesin.components.http.clj-http :as components.http]
[parenthesin.components.router.reitit-malli :as components.router]
[parenthesin.components.server.reitit-pedestal-jetty :as components.webserver]
[parenthesin.helpers.malli :as helpers.malli]
[parenthesin.helpers.state-flow.http :as state-flow.http]
[parenthesin.helpers.state-flow.server.pedestal :as state-flow.server]
Expand All @@ -18,20 +11,9 @@

(use-fixtures :once helpers.malli/with-intrumentation)

(defn- create-and-start-components! []
(component/start-system
(component/system-map
:config (components.config/new-config)
:http (components.http/new-http-mock {})
:router (components.router/new-router routes/routes)
:database (component/using (components.database/new-database)
[:config])
:webserver (component/using (components.webserver/new-webserver)
[:config :http :router :database]))))

(defflow
flow-integration-wallet-test
{:init (util/start-system! create-and-start-components!)
{:init util/start-system!
:cleanup util/stop-system!
:fail-fast? true}
(flow "should interact with system"
Expand Down
44 changes: 44 additions & 0 deletions test/unit/codes/clj/contest/submission_runner/adapters/submission_test.clj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
(ns unit.codes.clj.contest.submission-runner.adapters.submission-test
(:require [clj-commons.digest :as digest]
[clojure.test :refer [deftest is testing use-fixtures]]
[clojure.test.check.clojure-test :refer [defspec]]
[clojure.test.check.properties :as properties]
[codes.clj.contest.submission-runner.adapters.submission :as adapters.submission]
[codes.clj.contest.submission-runner.wire.db.submission :as wire.db.submission]
[codes.clj.contest.submission-runner.wire.in.submission :as wire.in.submission]
[malli.core :as m]
[malli.generator :as mg]
[matcher-combinators.test :refer [match?]]
[parenthesin.helpers.malli :as helpers.malli]))

(use-fixtures :once helpers.malli/with-intrumentation)

(defspec wirer->internal-spec 50
(properties/for-all [code (mg/generator wire.in.submission/Submission)]
(m/validate wire.db.submission/Submission (adapters.submission/wire->internal code))))

(def id (random-uuid))
(def code "(ns runner
(:require [clojure.string :as str]))
(defn my-sum [a b] (+ a b))")

(def code-submission {:id (str id)
:code code
:code-hash (digest/md5 code)
:language :clojure
:test-cases {:case-1 {:input "(my-sum 1 2)"
:output 3}
:case-2 {:input "(my-sum 2 3)"
:output 5}}})

(deftest wire->internal
(testing "adpater to db submission"
(is (match? {:submission/id (str id)
:submission/code code
:submission/code_hash (digest/md5 code)
:submission/language :clojure
:submission/test_cases {:case-1 {:input "(my-sum 1 2)"
:output 3}
:case-2 {:input "(my-sum 2 3)"
:output 5}}}
(adapters.submission/wire->internal code-submission)))))