Skip to content

refactor: remove kagglehub and add option to mount from drive #9452

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rasswanth-s merged 1 commit into from
Jul 17, 2026
+41 −28
Merged

refactor: remove kagglehub and add option to mount from drive #9452

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 41 additions & 28 deletions notebooks/enclave/gemma/colab/1. DO-model-owner-gemma-restrict.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,9 +209,7 @@
"id": "17",
"metadata": {},
"source": [
"## Step 2 — Download Gemma 3 weights from Kaggle\n",
"\n",
"Authenticate to Kaggle, then download. Weights are cached after first run."
"## Step 2 — Mount Gemma 3 Weights from Drive\n"
]
},
{
Expand All @@ -221,11 +219,8 @@
"metadata": {},
"outputs": [],
"source": [
"import kagglehub\n",
"\n",
"# Authenticate to Kaggle (one-time). You must also accept the Gemma license once at\n",
"# https://www.kaggle.com/models/google/gemma-3\n",
"kagglehub.login()"
"from google.colab import drive\n",
"drive.mount('/content/drive')"
]
},
{
Expand All @@ -235,16 +230,29 @@
"metadata": {},
"outputs": [],
"source": [
"print(f\"Downloading: {KAGGLE_HANDLE}\")\n",
"weights_dir = kagglehub.model_download(KAGGLE_HANDLE)\n",
"print(f\"Weights directory: {weights_dir}\")\n",
"print(f\"Contents: {os.listdir(weights_dir)}\")"
"from pathlib import Path\n",
"\n",
"GEMMA_3_WEIGHTS_DIR = Path(f\"~/.cache/kagglehub/models/google/gemma-3/flax\").expanduser().absolute()\n",
"GEMMA_3_WEIGHTS_DIR.mkdir(parents=True, exist_ok=True)\n",
"# copy drive weights to cache if not already present\n",
"!cp -R /content/drive/MyDrive/gemma-3-270m-it ~/.cache/kagglehub/models/google/gemma-3/flax\n",
"\n"
]
},
{
"cell_type": "markdown",
"cell_type": "code",
"execution_count": null,
"id": "20",
"metadata": {},
"outputs": [],
"source": [
"weights_dir = Path(f\"~/.cache/kagglehub/models/{KAGGLE_HANDLE}/{CKPT_SUBDIR}\").expanduser().absolute()"
]
},
{
"cell_type": "markdown",
"id": "21",
"metadata": {},
"source": [
"## Step 3 — Build the private dataset\n",
"\n",
Expand All @@ -259,7 +267,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "21",
"id": "22",
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -309,7 +317,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "22",
"id": "23",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -325,7 +333,7 @@
},
{
"cell_type": "markdown",
"id": "23",
"id": "24",
"metadata": {},
"source": [
"## Step 4 — Upload gemma3_model\n",
Expand All @@ -336,7 +344,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "24",
"id": "25",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -355,7 +363,7 @@
},
{
"cell_type": "markdown",
"id": "25",
"id": "26",
"metadata": {},
"source": [
"## Step 5 — Have the enclave run `syft-restrict` on the engine\n",
Expand All @@ -368,7 +376,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "26",
"id": "27",
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -428,7 +436,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "27",
"id": "28",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -449,7 +457,7 @@
},
{
"cell_type": "markdown",
"id": "28",
"id": "29",
"metadata": {},
"source": [
"### Step 5.1 — Approve it\n",
Expand All @@ -460,7 +468,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "29",
"id": "30",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -472,7 +480,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "30",
"id": "31",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -482,7 +490,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "31",
"id": "32",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -493,7 +501,7 @@
},
{
"cell_type": "markdown",
"id": "32",
"id": "33",
"metadata": {},
"source": [
"## Step 6 — Wait for the Researcher to submit the job, then approve\n",
Expand All @@ -504,7 +512,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "33",
"id": "34",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -517,7 +525,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "34",
"id": "35",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -527,7 +535,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "35",
"id": "36",
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -538,6 +546,11 @@
}
],
"metadata": {
"kernelspec": {
"display_name": "pysyft-enclave (3.12.8)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
Expand Down
Loading