train notebook

Signed-off-by: Max Pumperla <[email protected]>

Author: maxpumperla

_toc.yml

@@ -24,7 +24,6 @@ parts:
   - file: courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_02.ipynb
   - file: courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_03.ipynb
   - file: courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_04.ipynb
-  - file: courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_05.ipynb
   - file: courses/00_Developer_Intro_to_Ray/output/02b_Intro_Ray_Train_with_PyTorch_Lightning_01.ipynb
   - file: courses/00_Developer_Intro_to_Ray/output/02b_Intro_Ray_Train_with_PyTorch_Lightning_02.ipynb
   - file: courses/00_Developer_Intro_to_Ray/output/02b_Intro_Ray_Train_with_PyTorch_Lightning_03.ipynb

courses/00_Developer_Intro_to_Ray/02a_Intro_Ray_Train_with_PyTorch.ipynb

@@ -1332,7 +1332,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 4. Ray Train in Production\n",
+    "### 4. Ray Train in Production\n",
     "\n",
     "Here are some use-cases of using Ray Train in production:\n",
     "1. Canva uses Ray Train + Ray Data to cut down Stable Diffusion training costs by 3.7x. Read this [Anyscale blog post here](https://www.anyscale.com/blog/scalable-and-cost-efficient-stable-diffusion-pre-training-with-ray) and the [Canva  case study here](https://www.anyscale.com/resources/case-study/how-canva-built-a-modern-ai-platform-using-anyscale)\n",

courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_04.ipynb

@@ -767,6 +767,31 @@
     "\n",
     "</details>"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### 4. Ray Train in Production\n",
+    "\n",
+    "Here are some use-cases of using Ray Train in production:\n",
+    "1. Canva uses Ray Train + Ray Data to cut down Stable Diffusion training costs by 3.7x. Read this [Anyscale blog post here](https://www.anyscale.com/blog/scalable-and-cost-efficient-stable-diffusion-pre-training-with-ray) and the [Canva  case study here](https://www.anyscale.com/resources/case-study/how-canva-built-a-modern-ai-platform-using-anyscale)\n",
+    "2. Anyscale uses Ray Train + Deepspeed to finetune language models. Read more [here](https://github.com/ray-project/ray/tree/master/doc/source/templates/04_finetuning_llms_with_deepspeed).\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "# Run this cell for file cleanup \n",
+    "!rm -rf {storage_folder}/training/\n",
+    "!rm -rf {local_path}\n",
+    "!rm -rf {storage_folder}/data"
+   ]
   }
  ],
  "metadata": {

courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_05.ipynb

@@ -25,7 +25,6 @@ This collection contains multiple courses on Ray and Anyscale:
 - [02a Intro Ray Train with PyTorch 02](courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_02.ipynb)
 - [02a Intro Ray Train with PyTorch 03](courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_03.ipynb)
 - [02a Intro Ray Train with PyTorch 04](courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_04.ipynb)
-- [02a Intro Ray Train with PyTorch 05](courses/00_Developer_Intro_to_Ray/output/02a_Intro_Ray_Train_with_PyTorch_05.ipynb)
 - [02b Intro Ray Train with PyTorch Lightning 01](courses/00_Developer_Intro_to_Ray/output/02b_Intro_Ray_Train_with_PyTorch_Lightning_01.ipynb)
 - [02b Intro Ray Train with PyTorch Lightning 02](courses/00_Developer_Intro_to_Ray/output/02b_Intro_Ray_Train_with_PyTorch_Lightning_02.ipynb)
 - [02b Intro Ray Train with PyTorch Lightning 03](courses/00_Developer_Intro_to_Ray/output/02b_Intro_Ray_Train_with_PyTorch_Lightning_03.ipynb)