path: root/Computer_Vision_for_Industrial_Inspection/assessment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8a4a74dc-5958-4c8c-abdb-784135994d35",
   "metadata": {},
   "source": [
    "<a href=\"https://www.nvidia.com/dli\"><img src=\"images/DLI_Header.png\" alt=\"Header\" style=\"width: 400px;\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "571b7f4d-d7d6-4a80-8288-b6eb5d76901c",
   "metadata": {},
   "source": [
    "## Assessment: Computer Vision for Industrial Inspection ##\n",
    "In this notebook, you will utilize what you've learned in this course to complete an assessment. The assessment has been divided into a couple of steps to guide your development. You will be graded based on the performance of your deep learning model. Note that this coding portion does not give partial credit - it shows up as either 0 or 60 points. \n",
    "<table border=\"1\" class=\"dataframe\" align='left'>  <thead>    <tr style=\"text-align: right;\">      <th>Step</th>      <th></th>      <th>Points</th>    </tr>  </thead>  <tbody>    <tr>      <td>0. The Problem</td>      <td></td>      <td></td>    </tr>    <tr>      <td>1. Data Curation</td>      <td></td>      <td></td>    </tr>    <tr>      <td>2. Prepare TAO Experiment</td>      <td></td>      <td></td>    </tr>    <tr>      <td>3. Model Training</td>      <td></td>      <td></td>    </tr>    <tr>      <td>4. Model Evaluation</td>      <td></td>      <td>60</td>    </tr></tbody></table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "559c1a85-e36c-4396-93fb-524048c66684",
   "metadata": {},
   "source": [
    "<p><img src='images/ml_workflow.png' width=720></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43e4be02-6163-4282-ad82-892dee6d17a0",
   "metadata": {},
   "source": [
    "### Step 0: The Problem ###\n",
    "In this course, we made a binary classifier for the true/false defective units in our printed circuit board assembly dataset. For the asssesment we are asking you to create a model over the same dataset, but with a different purpose. Before we dealt with `capacitors` only, but the reference circuit boards have other components as well. In particular, we are interested in classifying the following four components based on their images: \n",
    "\n",
    "<p><img src='images/assessment_samples.png' width=720></p>\n",
    "\n",
    "The component types are marked by the first letter(s) of component identification numbers. \n",
    "* **C** - Capacitor\n",
    "* **Q** - Transistor\n",
    "* **R** - Resistor\n",
    "* **U** - Integrated Circuit\n",
    "\n",
    "Your task is to train and evaluate a classifier that accurately labels the four component types. We recommend using `VGG19` as the architecture in the spirit of experimenting with different backbones. \n",
    "\n",
    "**Instructions**: <br> \n",
    "0.1 Execute the below cell to import dependencies <br>\n",
    "0.2 Execute the cell below to unzip and load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6854c3b3-4640-4b5d-99dd-df66f1bb6226",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 0.1\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# import dependencies\n",
    "import os\n",
    "import warnings\n",
    "import pandas as pd\n",
    "import json\n",
    "import matplotlib.image as mpimg\n",
    "import matplotlib.pyplot as plt\n",
    "import math\n",
    "import shutil\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a3c9a2e4-9e8a-4031-b955-14a5c2f16f71",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 0.2\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# unzip\n",
    "!unzip -qq data/viz_BYD_new.zip -d data\n",
    "\n",
    "# remove zip file\n",
    "!rm data/viz_BYD_new.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac27c9b9-b8f9-4331-b6dd-f3a1c7da2058",
   "metadata": {},
   "source": [
    "### Step 1: Data Curation ###\n",
    "The first step is to curate the data. \n",
    "\n",
    "**Instructions**: <br>\n",
    "1.1 Execute the below cell to load the data into a `DataFrame` and preview. <br>\n",
    "1.2 Modify the `<FIXME>` only and execute the cell below to filter and keep only normal images as potential defective images may include missing parts. <br>\n",
    "1.3 Modify the `<FIXME>` only and execute the cell below to filter and keep only relevant images, i.e. `C`, `Q`, `R`, or `U`. <br>\n",
    "1.4 Execute the cell below to check the sample size. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "47f1a7e0-ed48-4378-8ddb-56d9ef8a1357",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>true_defect</th>\n",
       "      <th>defect_img_path</th>\n",
       "      <th>date</th>\n",
       "      <th>board</th>\n",
       "      <th>comp_id</th>\n",
       "      <th>img_shape</th>\n",
       "      <th>defect_image_name</th>\n",
       "      <th>comp_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>notdefect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423318026324/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423318026324</td>\n",
       "      <td>C1090</td>\n",
       "      <td>(54, 27, 3)</td>\n",
       "      <td>D0_C1090.jpg</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>notdefect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423318026269/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423318026269</td>\n",
       "      <td>C1090</td>\n",
       "      <td>(54, 27, 3)</td>\n",
       "      <td>D1_C1090.jpg</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>notdefect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423318026261/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423318026261</td>\n",
       "      <td>L503</td>\n",
       "      <td>(347, 418, 3)</td>\n",
       "      <td>D1_L503.jpg</td>\n",
       "      <td>L</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>notdefect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423318026523/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423318026523</td>\n",
       "      <td>L503</td>\n",
       "      <td>(347, 418, 3)</td>\n",
       "      <td>D1_L503.jpg</td>\n",
       "      <td>L</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>notdefect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423318026523/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423318026523</td>\n",
       "      <td>C1090</td>\n",
       "      <td>(54, 27, 3)</td>\n",
       "      <td>D1_C1090.jpg</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  true_defect                                    defect_img_path  date  \\\n",
       "0   notdefect  /dli/task/data/AOI_DL_data_0908/0423318026324/...   908   \n",
       "1   notdefect  /dli/task/data/AOI_DL_data_0908/0423318026269/...   908   \n",
       "2   notdefect  /dli/task/data/AOI_DL_data_0908/0423318026261/...   908   \n",
       "3   notdefect  /dli/task/data/AOI_DL_data_0908/0423318026523/...   908   \n",
       "4   notdefect  /dli/task/data/AOI_DL_data_0908/0423318026523/...   908   \n",
       "\n",
       "          board comp_id      img_shape defect_image_name comp_type  \n",
       "0  423318026324   C1090    (54, 27, 3)      D0_C1090.jpg         C  \n",
       "1  423318026269   C1090    (54, 27, 3)      D1_C1090.jpg         C  \n",
       "2  423318026261    L503  (347, 418, 3)       D1_L503.jpg         L  \n",
       "3  423318026523    L503  (347, 418, 3)       D1_L503.jpg         L  \n",
       "4  423318026523   C1090    (54, 27, 3)      D1_C1090.jpg         C  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1.1\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# load from pcba_df.csv\n",
    "pcba_df=pd.read_csv('pcba_df.csv')\n",
    "pcba_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "1831b93c-deb5-40c6-a55b-4feb22c568fa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>true_defect</th>\n",
       "      <th>defect_img_path</th>\n",
       "      <th>date</th>\n",
       "      <th>board</th>\n",
       "      <th>comp_id</th>\n",
       "      <th>img_shape</th>\n",
       "      <th>defect_image_name</th>\n",
       "      <th>comp_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618041671/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618041671</td>\n",
       "      <td>R511</td>\n",
       "      <td>(132, 52, 3)</td>\n",
       "      <td>D1_R511.jpg</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618042253/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618042253</td>\n",
       "      <td>R1318</td>\n",
       "      <td>(88, 35, 3)</td>\n",
       "      <td>D1_R1318.jpg</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618039275/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618039275</td>\n",
       "      <td>C517</td>\n",
       "      <td>(387, 171, 3)</td>\n",
       "      <td>D1_C517.jpg</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>225</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618040901/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618040901</td>\n",
       "      <td>R176</td>\n",
       "      <td>(36, 87, 3)</td>\n",
       "      <td>D0_R176.jpg</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>227</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618039308/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618039308</td>\n",
       "      <td>C107</td>\n",
       "      <td>(84, 32, 3)</td>\n",
       "      <td>D1_C107.jpg</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    true_defect                                    defect_img_path  date  \\\n",
       "77       defect  /dli/task/data/AOI_DL_data_0908/0423618041671/...   908   \n",
       "89       defect  /dli/task/data/AOI_DL_data_0908/0423618042253/...   908   \n",
       "122      defect  /dli/task/data/AOI_DL_data_0908/0423618039275/...   908   \n",
       "225      defect  /dli/task/data/AOI_DL_data_0908/0423618040901/...   908   \n",
       "227      defect  /dli/task/data/AOI_DL_data_0908/0423618039308/...   908   \n",
       "\n",
       "            board comp_id      img_shape defect_image_name comp_type  \n",
       "77   423618041671    R511   (132, 52, 3)       D1_R511.jpg         R  \n",
       "89   423618042253   R1318    (88, 35, 3)      D1_R1318.jpg         R  \n",
       "122  423618039275    C517  (387, 171, 3)       D1_C517.jpg         C  \n",
       "225  423618040901    R176    (36, 87, 3)       D0_R176.jpg         R  \n",
       "227  423618039308    C107    (84, 32, 3)       D1_C107.jpg         C  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1.2\n",
    "verified_df=pcba_df[pcba_df['true_defect']==\"defect\"]\n",
    "verified_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ecb4e9ed-2d53-4d79-b67e-9e6da8fe2106",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>true_defect</th>\n",
       "      <th>defect_img_path</th>\n",
       "      <th>date</th>\n",
       "      <th>board</th>\n",
       "      <th>comp_id</th>\n",
       "      <th>img_shape</th>\n",
       "      <th>defect_image_name</th>\n",
       "      <th>comp_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618041671/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618041671</td>\n",
       "      <td>R511</td>\n",
       "      <td>(132, 52, 3)</td>\n",
       "      <td>D1_R511.jpg</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618042253/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618042253</td>\n",
       "      <td>R1318</td>\n",
       "      <td>(88, 35, 3)</td>\n",
       "      <td>D1_R1318.jpg</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618039275/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618039275</td>\n",
       "      <td>C517</td>\n",
       "      <td>(387, 171, 3)</td>\n",
       "      <td>D1_C517.jpg</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>225</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618040901/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618040901</td>\n",
       "      <td>R176</td>\n",
       "      <td>(36, 87, 3)</td>\n",
       "      <td>D0_R176.jpg</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>227</th>\n",
       "      <td>defect</td>\n",
       "      <td>/dli/task/data/AOI_DL_data_0908/0423618039308/...</td>\n",
       "      <td>908</td>\n",
       "      <td>423618039308</td>\n",
       "      <td>C107</td>\n",
       "      <td>(84, 32, 3)</td>\n",
       "      <td>D1_C107.jpg</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    true_defect                                    defect_img_path  date  \\\n",
       "77       defect  /dli/task/data/AOI_DL_data_0908/0423618041671/...   908   \n",
       "89       defect  /dli/task/data/AOI_DL_data_0908/0423618042253/...   908   \n",
       "122      defect  /dli/task/data/AOI_DL_data_0908/0423618039275/...   908   \n",
       "225      defect  /dli/task/data/AOI_DL_data_0908/0423618040901/...   908   \n",
       "227      defect  /dli/task/data/AOI_DL_data_0908/0423618039308/...   908   \n",
       "\n",
       "            board comp_id      img_shape defect_image_name comp_type  \n",
       "77   423618041671    R511   (132, 52, 3)       D1_R511.jpg         R  \n",
       "89   423618042253   R1318    (88, 35, 3)      D1_R1318.jpg         R  \n",
       "122  423618039275    C517  (387, 171, 3)       D1_C517.jpg         C  \n",
       "225  423618040901    R176    (36, 87, 3)       D0_R176.jpg         R  \n",
       "227  423618039308    C107    (84, 32, 3)       D1_C107.jpg         C  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1.3\n",
    "cqru_df=verified_df[verified_df['comp_type'].isin([\"C\", \"Q\", \"R\", \"U\"])]\n",
    "cqru_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fa67e90e-876c-4280-81d6-13452b6bae82",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "comp_type\n",
       "C    99\n",
       "R    12\n",
       "U     8\n",
       "dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1.4\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# check sample size\n",
    "cqru_df.groupby('comp_type').size()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c569654d-7fe0-4af4-b8ea-d2326bb684a2",
   "metadata": {},
   "source": [
    "### Step 2: Prepare TAO Experiment ###\n",
    "Next we will prepare for the TAO experiment. \n",
    "\n",
    "**Instructions**: <br>\n",
    "2.1 Execute the below cell to set environment variables. <br>\n",
    "2.2 Execute the cell below to map up local directories to the TAO docker. <br>\n",
    "2.3 Execute the cell below to use the `ngc registry mode list` command that lists all available `classification` models. <br>\n",
    "2.4 Modify the `<FIXME>` only and execute the cell below to download the `VGG19` pre-trained weights. <br>\n",
    "2.5 Execute the cell below to view the pre-trained model. <br>\n",
    "2.6 Execute the cell below to create the required data directories. <br>\n",
    "2.7 Execute the cell below to create a 70%/30% split for train and val. <br>\n",
    "2.8 Execute the cell below to copy data from the source to the TAO experiment folder. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e55f23d5-5509-4a22-8d8b-b2c9612d75c3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: KEY=my_model_key\n",
      "env: LOCAL_PROJECT_DIR=/dli/task/tao_project\n",
      "env: LOCAL_DATA_DIR=/dli/task/tao_project/data\n",
      "env: LOCAL_SPECS_DIR=/dli/task/tao_project/spec_files\n",
      "env: TAO_PROJECT_DIR=/workspace/tao-experiments\n",
      "env: TAO_DATA_DIR=/workspace/tao-experiments/data\n",
      "env: TAO_SPECS_DIR=/workspace/tao-experiments/spec_files\n"
     ]
    }
   ],
   "source": [
    "# 2.1\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# set environment variables\n",
    "%set_env KEY=my_model_key\n",
    "\n",
    "%set_env LOCAL_PROJECT_DIR=/dli/task/tao_project\n",
    "%set_env LOCAL_DATA_DIR=/dli/task/tao_project/data\n",
    "%set_env LOCAL_SPECS_DIR=/dli/task/tao_project/spec_files\n",
    "os.environ[\"LOCAL_EXPERIMENT_DIR\"]=os.path.join(os.getenv(\"LOCAL_PROJECT_DIR\"), \"classification\")\n",
    "\n",
    "%set_env TAO_PROJECT_DIR=/workspace/tao-experiments\n",
    "%set_env TAO_DATA_DIR=/workspace/tao-experiments/data\n",
    "%set_env TAO_SPECS_DIR=/workspace/tao-experiments/spec_files\n",
    "os.environ['TAO_EXPERIMENT_DIR']=os.path.join(os.getenv(\"TAO_PROJECT_DIR\"), \"classification\")\n",
    "\n",
    "# make the data directory\n",
    "!mkdir -p $LOCAL_DATA_DIR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "bb3a6cbe-430d-4c54-8897-64aa5d41a2b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2.2\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# map local directories to the TAO docker\n",
    "mounts_file = os.path.expanduser(\"~/.tao_mounts.json\")\n",
    "\n",
    "drive_map = {\n",
    "    \"Mounts\": [\n",
    "            # Mapping the data directory\n",
    "            {\n",
    "                \"source\": os.environ[\"LOCAL_PROJECT_DIR\"],\n",
    "                \"destination\": \"/workspace/tao-experiments\"\n",
    "            },\n",
    "            # Mapping the specs directory.\n",
    "            {\n",
    "                \"source\": os.environ[\"LOCAL_SPECS_DIR\"],\n",
    "                \"destination\": os.environ[\"TAO_SPECS_DIR\"]\n",
    "            },\n",
    "            # Mapping the data directory.\n",
    "            {\n",
    "                \"source\": os.environ[\"LOCAL_DATA_DIR\"],\n",
    "                \"destination\": os.environ[\"TAO_DATA_DIR\"]\n",
    "            },\n",
    "        ],\n",
    "    \"DockerOptions\": {\n",
    "        \"user\": \"{}:{}\".format(os.getuid(), os.getgid())\n",
    "    }\n",
    "}\n",
    "\n",
    "# writing the mounts file\n",
    "with open(mounts_file, \"w\") as mfile:\n",
    "    json.dump(drive_map, mfile, indent=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0ebc8c2a-bb02-4515-8923-1cb6f53b2bef",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    \"versionId\": \"vgg19\"\n"
     ]
    }
   ],
   "source": [
    "# 2.3\n",
    "# DO NOT CHANGE THIS CELL\n",
    "!ngc registry model list nvidia/tao/pretrained_classification:* | grep vgg19"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "548bd5b4-0e54-48f7-82d1-549623cd92db",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"download_end\": \"2025-12-19 11:01:35\",\n",
      "    \"download_start\": \"2025-12-19 11:01:31\",\n",
      "    \"download_time\": \"4s\",\n",
      "    \"files_downloaded\": 1,\n",
      "    \"local_path\": \"/dli/task/tao_project/classification/pretrained_vgg19/pretrained_classification_vvgg19\",\n",
      "    \"size_downloaded\": \"153.72 MB\",\n",
      "    \"status\": \"COMPLETED\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# 2.4\n",
    "# create directory to store the pre-trained model\n",
    "!mkdir -p $LOCAL_EXPERIMENT_DIR/pretrained_vgg19/\n",
    "\n",
    "# download the pre-trained vgg19 model from NGC\n",
    "!ngc registry model download-version nvidia/tao/pretrained_classification:vgg19 --dest $LOCAL_EXPERIMENT_DIR/pretrained_vgg19"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1cc0cf5-4c9d-476e-a302-6a175fc04d3a",
   "metadata": {},
   "source": [
    "<p><img src='images/tip.png' width=720></p>\n",
    "\n",
    "We designated the model to be downloaded to `tao_project/classification/pretrained_vgg19`, which is mapped to `/workspace/tao-experiments/classification/pretrained_vgg19` in the TAO container based on the mapping of `LOCAL_EXPERIMENT_DIR` to `TAO_EXPERIMENT_DIR`. Looking at the `local_path` and `transfer_id` keys of the output JSON, we can gather that the path of the pre-trained model should be in the `tao_project/classification/pretrained_vgg19/pretrained_classification_vvgg19` directory. When referencing paths for the TAO Toolkit, it's important to use paths based on the TAO container. In this case it would be `/workspace/tao-experiments/classification/pretrained_vgg19/pretrained_classification_vvgg19`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "e7655abf-8c17-4c85-86cb-8f594e932f61",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 157416\n",
      "drwxr-xr-x 2 root root      4096 Dec 19 11:01 .\n",
      "drwxr-xr-x 3 root root      4096 Dec 19 11:01 ..\n",
      "-rw-r--r-- 1 root root 161183816 Dec 19 11:01 vgg_19.hdf5\n"
     ]
    }
   ],
   "source": [
    "# 2.5\n",
    "# DO NOT CHANGE THIS CELL\n",
    "!ls -al tao_project/classification/pretrained_vgg19/pretrained_classification_vvgg19"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "2538dd6b-51bb-4347-b48b-fe1c1ee69dda",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2.6\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# remove existing data from previous experiment (if any)\n",
    "!rm -rf $LOCAL_DATA_DIR/*\n",
    "\n",
    "!mkdir -p $LOCAL_DATA_DIR/train/Q\n",
    "!mkdir -p $LOCAL_DATA_DIR/train/C\n",
    "!mkdir -p $LOCAL_DATA_DIR/train/U\n",
    "!mkdir -p $LOCAL_DATA_DIR/train/R\n",
    "!mkdir -p $LOCAL_DATA_DIR/val/Q\n",
    "!mkdir -p $LOCAL_DATA_DIR/val/C\n",
    "!mkdir -p $LOCAL_DATA_DIR/val/U\n",
    "!mkdir -p $LOCAL_DATA_DIR/val/R"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "1521c5f5-3f0c-461e-a561-664dc68d8ef3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "data_set  comp_type\n",
       "train     C            69\n",
       "          R             8\n",
       "          U             6\n",
       "val       C            30\n",
       "          R             4\n",
       "          U             2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 2.7\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# set default as training set\n",
    "cqru_df['data_set']='train'\n",
    "\n",
    "# sample 30% and set as validation set\n",
    "val_set=cqru_df.groupby('comp_type', group_keys=False).apply(lambda x: x.sample(frac=0.3))\n",
    "cqru_df.loc[val_set.index, 'data_set']='val'\n",
    "cqru_df.groupby(['data_set', 'comp_type']).size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "65f67e2b-0b15-4264-bc71-f08423236ff0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2.8\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# iterate through the DataFrame and copy images\n",
    "for idx, row in cqru_df.iterrows(): \n",
    "    shutil.copyfile(row['defect_img_path'], f\"{os.environ['LOCAL_DATA_DIR']}/{row['data_set']}/{row['comp_type']}/{row['date']}_{row['board']}_{row['defect_image_name']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "455daecb-fbd1-4e1e-a831-68b4672f10ed",
   "metadata": {},
   "source": [
    "### Step 3: Model Training ###\n",
    "The next step is to modify the configuration file that will be used for `train`. You can create a new text file for this purpose manually and start from scratch or use the [template provided](tao_project/spec_files/vgg19/config.txt). You can also refer to sample applications and configuration files [here](https://docs.nvidia.com/tao/tao-toolkit/text/image_classification.html). \n",
    "\n",
    "**Instructions**: <br>\n",
    "3.1. Open and review the [configuration file](tao_project/spec_files/vgg19/config.txt). <br>\n",
    "3.2. Modify the `<FIXME>`s only in the configuration file with the correct values and **save changes**. We recommend starting with a very low epoch count (e.g. 5) in the interest of time as each epoch can take ~100s to complete. <br>\n",
    "* Recall that you don't need the `eval_config` section of the configuration file, which requires you to know the trained model path. Of course, if you know where you would like to place the model, you can go ahead and complete this section. \n",
    "3.3 Execute the cell below to initiate model training. \n",
    "\n",
    "<p><img src='images/tip.png' width=720></p>\n",
    "\n",
    "Based on how NGC names the pre-trained model downloaded, we should use `/workspace/tao-experiments/classification/pretrained_vgg19/pretrained_classification_vvgg19/resnet_19.hdf5` to reference the pre-trained model. Furthermore, we can choose where to store the trained model - in this case we use `/workspace/tao-experiments/classification/vgg19` inside of the TAO container, which is mapped to `tao_project/classification/vgg19` in our local drive. Furthermore, the trained model name will follow the format `<model_arch>_0<last_epoch_count>.tlt`, unless specified otherwise. Therefore we should use `/workspace/tao-experiments/classification/vgg19/weights/vgg_XXX.tlt`, depending on `n_epochs` in the model configuration. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "b3fba5b6-ee35-48ab-9f09-6608c9388685",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2025-12-19 11:11:38,782 [TAO Toolkit] [INFO] root 160: Registry: ['nvcr.io']\n",
      "2025-12-19 11:11:38,891 [TAO Toolkit] [INFO] nvidia_tao_cli.components.instance_handler.local_instance 360: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5\n",
      "2025-12-19 11:11:38,903 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 301: Printing tty value True\n",
      "Using TensorFlow backend.\n",
      "2025-12-19 11:11:39.898674: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12\n",
      "2025-12-19 11:11:39,950 [TAO Toolkit] [WARNING] tensorflow 40: Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.\n",
      "2025-12-19 11:11:41,183 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "2025-12-19 11:11:41,225 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "2025-12-19 11:11:41,230 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:150: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_hue(img, max_delta=10.0):\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:173: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_saturation(img, max_shift):\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:183: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_contrast(img, center, max_contrast_scale):\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:192: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_shift(x_img, shift_stddev):\n",
      "2025-12-19 11:11:43.235732: I tensorflow/core/platform/profile_utils/cpu_utils.cc:109] CPU Frequency: 2499995000 Hz\n",
      "2025-12-19 11:11:43.236164: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x89d3ab0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:\n",
      "2025-12-19 11:11:43.236201: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version\n",
      "2025-12-19 11:11:43.237998: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcuda.so.1\n",
      "2025-12-19 11:11:43.431269: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:11:43.433367: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x87eee60 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:\n",
      "2025-12-19 11:11:43.433407: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla T4, Compute Capability 7.5\n",
      "2025-12-19 11:11:43.433772: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:11:43.435634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1674] Found device 0 with properties: \n",
      "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
      "pciBusID: 0000:00:1e.0\n",
      "2025-12-19 11:11:43.435695: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12\n",
      "2025-12-19 11:11:43.435814: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcublas.so.12\n",
      "2025-12-19 11:11:43.438077: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcufft.so.11\n",
      "2025-12-19 11:11:43.438213: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcurand.so.10\n",
      "2025-12-19 11:11:43.441791: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusolver.so.11\n",
      "2025-12-19 11:11:43.443000: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusparse.so.12\n",
      "2025-12-19 11:11:43.443087: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudnn.so.8\n",
      "2025-12-19 11:11:43.443250: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:11:43.445203: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:11:43.447008: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1802] Adding visible gpu devices: 0\n",
      "2025-12-19 11:11:43.447066: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12\n",
      "2025-12-19 11:11:43.456651: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1214] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
      "2025-12-19 11:11:43.456703: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1220]      0 \n",
      "2025-12-19 11:11:43.456713: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1233] 0:   N \n",
      "2025-12-19 11:11:43.457035: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:11:43.459060: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:11:43.460917: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1359] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 13496 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:1e.0, compute capability: 7.5)\n",
      "Using TensorFlow backend.\n",
      "WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.\n",
      "WARNING:tensorflow:TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "2025-12-19 11:11:45,088 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "WARNING:tensorflow:TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "2025-12-19 11:11:45,127 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "WARNING:tensorflow:TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "2025-12-19 11:11:45,131 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:150: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_hue(img, max_delta=10.0):\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:173: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_saturation(img, max_shift):\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:183: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_contrast(img, center, max_contrast_scale):\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:192: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_shift(x_img, shift_stddev):\n",
      "2025-12-19 11:11:46,392 [TAO Toolkit] [INFO] __main__ 388: Loading experiment spec at /workspace/tao-experiments/spec_files/vgg19/config.txt.\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/train.py:398: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n",
      "\n",
      "2025-12-19 11:11:46,395 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/train.py:398: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/train.py:407: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
      "\n",
      "2025-12-19 11:11:46,395 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/train.py:407: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
      "\n",
      "2025-12-19 11:11:46,739 [TAO Toolkit] [INFO] nvidia_tao_tf1.cv.common.logging.logging 197: Log file already exists at /workspace/tao-experiments/classification/vgg19/status.json\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/train.py:431: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.\n",
      "\n",
      "2025-12-19 11:11:46,739 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/train.py:431: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/train.py:431: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.\n",
      "\n",
      "2025-12-19 11:11:46,740 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/train.py:431: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.\n",
      "\n",
      "2025-12-19 11:11:46,740 [TAO Toolkit] [INFO] __main__ 478: Default image mean [103.939, 116.779, 123.68] will be used.\n",
      "Found 83 images belonging to 4 classes.\n",
      "2025-12-19 11:11:46,749 [TAO Toolkit] [INFO] __main__ 294: Processing dataset (train): /workspace/tao-experiments/data/train\n",
      "Found 36 images belonging to 4 classes.\n",
      "2025-12-19 11:11:46,752 [TAO Toolkit] [INFO] __main__ 311: Processing dataset (validation): /workspace/tao-experiments/data/val\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n",
      "\n",
      "2025-12-19 11:11:46,753 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n",
      "\n",
      "2025-12-19 11:11:46,753 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n",
      "\n",
      "2025-12-19 11:11:46,756 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/third_party/keras/tensorflow_backend.py:199: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.\n",
      "\n",
      "2025-12-19 11:11:46,858 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/third_party/keras/tensorflow_backend.py:199: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.\n",
      "\n",
      "2025-12-19 11:11:47,021 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:190: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n",
      "\n",
      "2025-12-19 11:11:47,021 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:190: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:199: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead.\n",
      "\n",
      "2025-12-19 11:11:47,021 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:199: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:206: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.\n",
      "\n",
      "2025-12-19 11:11:47,102 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:206: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:1834: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.\n",
      "\n",
      "2025-12-19 11:11:47,474 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:1834: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:133: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.\n",
      "\n",
      "2025-12-19 11:11:47,480 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:133: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.\n",
      "\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         (None, 3, 224, 224)       0         \n",
      "_________________________________________________________________\n",
      "block_1a_conv_1 (Conv2D)     (None, 64, 224, 224)      1728      \n",
      "_________________________________________________________________\n",
      "block_1a_relu (Activation)   (None, 64, 224, 224)      0         \n",
      "_________________________________________________________________\n",
      "block_1b_conv_1 (Conv2D)     (None, 64, 224, 224)      36864     \n",
      "_________________________________________________________________\n",
      "block_1b_relu (Activation)   (None, 64, 224, 224)      0         \n",
      "_________________________________________________________________\n",
      "block_2a_conv_1 (Conv2D)     (None, 128, 112, 112)     73728     \n",
      "_________________________________________________________________\n",
      "block_2a_relu (Activation)   (None, 128, 112, 112)     0         \n",
      "_________________________________________________________________\n",
      "block_2b_conv_1 (Conv2D)     (None, 128, 112, 112)     147456    \n",
      "_________________________________________________________________\n",
      "block_2b_relu (Activation)   (None, 128, 112, 112)     0         \n",
      "_________________________________________________________________\n",
      "block_3a_conv_1 (Conv2D)     (None, 256, 56, 56)       294912    \n",
      "_________________________________________________________________\n",
      "block_3a_relu (Activation)   (None, 256, 56, 56)       0         \n",
      "_________________________________________________________________\n",
      "block_3b_conv_1 (Conv2D)     (None, 256, 56, 56)       589824    \n",
      "_________________________________________________________________\n",
      "block_3b_relu (Activation)   (None, 256, 56, 56)       0         \n",
      "_________________________________________________________________\n",
      "block_3c_conv_1 (Conv2D)     (None, 256, 56, 56)       589824    \n",
      "_________________________________________________________________\n",
      "block_3c_relu (Activation)   (None, 256, 56, 56)       0         \n",
      "_________________________________________________________________\n",
      "block_3d_conv_1 (Conv2D)     (None, 256, 56, 56)       589824    \n",
      "_________________________________________________________________\n",
      "block_3d_relu (Activation)   (None, 256, 56, 56)       0         \n",
      "_________________________________________________________________\n",
      "block_4a_conv_1 (Conv2D)     (None, 512, 28, 28)       1179648   \n",
      "_________________________________________________________________\n",
      "block_4a_relu (Activation)   (None, 512, 28, 28)       0         \n",
      "_________________________________________________________________\n",
      "block_4b_conv_1 (Conv2D)     (None, 512, 28, 28)       2359296   \n",
      "_________________________________________________________________\n",
      "block_4b_relu (Activation)   (None, 512, 28, 28)       0         \n",
      "_________________________________________________________________\n",
      "block_4c_conv_1 (Conv2D)     (None, 512, 28, 28)       2359296   \n",
      "_________________________________________________________________\n",
      "block_4c_relu (Activation)   (None, 512, 28, 28)       0         \n",
      "_________________________________________________________________\n",
      "block_4d_conv_1 (Conv2D)     (None, 512, 28, 28)       2359296   \n",
      "_________________________________________________________________\n",
      "block_4d_relu (Activation)   (None, 512, 28, 28)       0         \n",
      "_________________________________________________________________\n",
      "block_5a_conv_1 (Conv2D)     (None, 512, 14, 14)       2359296   \n",
      "_________________________________________________________________\n",
      "block_5a_relu (Activation)   (None, 512, 14, 14)       0         \n",
      "_________________________________________________________________\n",
      "block_5b_conv_1 (Conv2D)     (None, 512, 14, 14)       2359296   \n",
      "_________________________________________________________________\n",
      "block_5b_relu (Activation)   (None, 512, 14, 14)       0         \n",
      "_________________________________________________________________\n",
      "block_5c_conv_1 (Conv2D)     (None, 512, 14, 14)       2359296   \n",
      "_________________________________________________________________\n",
      "block_5c_relu (Activation)   (None, 512, 14, 14)       0         \n",
      "_________________________________________________________________\n",
      "block_5d_conv_1 (Conv2D)     (None, 512, 14, 14)       2359296   \n",
      "_________________________________________________________________\n",
      "block_5d_relu (Activation)   (None, 512, 14, 14)       0         \n",
      "_________________________________________________________________\n",
      "avg_pool (AveragePooling2D)  (None, 512, 1, 1)         0         \n",
      "_________________________________________________________________\n",
      "flatten (Flatten)            (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "predictions (Dense)          (None, 4)                 2052      \n",
      "=================================================================\n",
      "Total params: 20,020,932\n",
      "Trainable params: 20,020,932\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n",
      "\n",
      "2025-12-19 11:11:51,055 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3295: The name tf.log is deprecated. Please use tf.math.log instead.\n",
      "\n",
      "2025-12-19 11:11:51,067 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:3295: The name tf.log is deprecated. Please use tf.math.log instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/common/utils.py:1133: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n",
      "2025-12-19 11:11:51,082 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/common/utils.py:1133: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n",
      "2025-12-19 11:11:51,083 [TAO Toolkit] [INFO] nvidia_tao_tf1.cv.common.logging.logging 197: Log file already exists at /workspace/tao-experiments/classification/vgg19/status.json\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:986: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead.\n",
      "\n",
      "2025-12-19 11:11:51,492 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:986: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:973: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.\n",
      "\n",
      "2025-12-19 11:11:51,539 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:973: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/common/utils.py:1181: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n",
      "2025-12-19 11:11:52,036 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/common/utils.py:1181: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
      "\n",
      "2025-12-19 11:11:52,337 [TAO Toolkit] [INFO] root 2102: Starting Training Loop.\n",
      "Epoch 1/5\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/common/utils.py:199: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n",
      "2025-12-19 11:12:31,657 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/common/utils.py:199: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n",
      "3/3 [==============================] - 104s 35s/step - loss: 16.1038 - acc: 0.0162 - val_loss: 15.4765 - val_acc: 0.0556\n",
      "2025-12-19 11:13:39,077 [TAO Toolkit] [INFO] root 2102: Training loop in progress\n",
      "Epoch 2/5\n",
      "3/3 [==============================] - 19s 6s/step - loss: 15.2391 - acc: 0.0703 - val_loss: 15.4769 - val_acc: 0.0556\n",
      "2025-12-19 11:13:59,330 [TAO Toolkit] [INFO] root 2102: Training loop in progress\n",
      "Epoch 3/5\n",
      "3/3 [==============================] - 2s 560ms/step - loss: 15.1533 - acc: 0.0756 - val_loss: 15.4773 - val_acc: 0.0556\n",
      "2025-12-19 11:14:01,335 [TAO Toolkit] [INFO] root 2102: Training loop in progress\n",
      "Epoch 4/5\n",
      "3/3 [==============================] - 2s 559ms/step - loss: 15.2399 - acc: 0.0703 - val_loss: 15.4777 - val_acc: 0.0556\n",
      "2025-12-19 11:14:03,208 [TAO Toolkit] [INFO] root 2102: Training loop in progress\n",
      "Epoch 5/5\n",
      "3/3 [==============================] - 2s 556ms/step - loss: 15.1541 - acc: 0.0756 - val_loss: 15.4780 - val_acc: 0.0556\n",
      "2025-12-19 11:14:05,091 [TAO Toolkit] [INFO] root 2102: Training loop in progress\n",
      "2025-12-19 11:14:05,100 [TAO Toolkit] [INFO] root 2102: Training loop complete.\n",
      "2025-12-19 11:14:05,100 [TAO Toolkit] [INFO] root 2102: Final model evaluation in progress.\n",
      "2025-12-19 11:14:05,575 [TAO Toolkit] [INFO] root 2102: Model evaluation is complete.\n",
      "2025-12-19 11:14:05,575 [TAO Toolkit] [INFO] __main__ 625: Total Val Loss: 15.478020668029785\n",
      "2025-12-19 11:14:05,575 [TAO Toolkit] [INFO] __main__ 626: Total Val accuracy: 0.0555555559694767\n",
      "2025-12-19 11:14:05,575 [TAO Toolkit] [INFO] root 2102: Training finished successfully.\n",
      "2025-12-19 11:14:05,576 [TAO Toolkit] [INFO] __main__ 651: Training finished successfully.\n",
      "Telemetry data couldn't be sent, but the command ran successfully.\n",
      "[WARNING]: <urlopen error Url for the certificates not found.>\n",
      "Execution status: PASS\n",
      "2025-12-19 11:14:27,801 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 363: Stopping container.\n"
     ]
    }
   ],
   "source": [
    "# 3.2\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# train model\n",
    "!tao model classification_tf1 train -e $TAO_SPECS_DIR/vgg19/config.txt \\\n",
    "                                   -r $TAO_EXPERIMENT_DIR/vgg19 \\\n",
    "                                   -k $KEY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "9508c989-db0f-4f67-a2c0-bb36127e5e07",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 765M\n",
      "-rw-r--r-- 1 root root 153M Dec 19 11:13 vgg_001.hdf5\n",
      "-rw-r--r-- 1 root root 153M Dec 19 11:13 vgg_002.hdf5\n",
      "-rw-r--r-- 1 root root 153M Dec 19 11:14 vgg_003.hdf5\n",
      "-rw-r--r-- 1 root root 153M Dec 19 11:14 vgg_004.hdf5\n",
      "-rw-r--r-- 1 root root 153M Dec 19 11:14 vgg_005.hdf5\n"
     ]
    }
   ],
   "source": [
    "!ls -ltrh $LOCAL_EXPERIMENT_DIR/vgg19/weights/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96d92840-c3cd-4b9a-b025-c47d1bbb2f32",
   "metadata": {},
   "source": [
    "### Step 4: Model Evaluation ###\n",
    "The last step for the assessment is to `evaluate` the model through the `eval_config` section of the configuration file. Once completed, you should submit the output log for assessment. \n",
    "\n",
    "**Instructions**: <br>\n",
    "4.1 Review the [configuration file](tao_project/spec_files/vgg19/config.txt) and modify the `eval_config` section if needed. <br>\n",
    "4.2 Execute the below cell to evaluate the model. A log file will be generated through the use of the `--log_file` option, which will be used for grading purposes. <br>\n",
    "4.3 Execute the cell below to submit the log for assessment. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "b637cba7-5394-4671-b6c9-49df0ff7303d",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2025-12-19 11:20:47,502 [TAO Toolkit] [INFO] root 160: Registry: ['nvcr.io']\n",
      "2025-12-19 11:20:47,618 [TAO Toolkit] [INFO] nvidia_tao_cli.components.instance_handler.local_instance 360: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5\n",
      "2025-12-19 11:20:47,638 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 301: Printing tty value True\n",
      "Using TensorFlow backend.\n",
      "2025-12-19 11:20:48.792442: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12\n",
      "2025-12-19 11:20:48,851 [TAO Toolkit] [WARNING] tensorflow 40: Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.\n",
      "2025-12-19 11:20:51,336 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "2025-12-19 11:20:51,471 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "2025-12-19 11:20:51,487 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable  TF_ALLOW_IOLIBS=1.\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:150: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_hue(img, max_delta=10.0):\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:173: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_saturation(img, max_shift):\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:183: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_contrast(img, center, max_contrast_scale):\n",
      "/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:192: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.\n",
      "  def random_shift(x_img, shift_stddev):\n",
      "2025-12-19 11:20:54.267904: I tensorflow/core/platform/profile_utils/cpu_utils.cc:109] CPU Frequency: 2499995000 Hz\n",
      "2025-12-19 11:20:54.268335: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x87e8de0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:\n",
      "2025-12-19 11:20:54.268369: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version\n",
      "2025-12-19 11:20:54.270032: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcuda.so.1\n",
      "2025-12-19 11:20:54.462303: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:20:54.464403: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x8604190 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:\n",
      "2025-12-19 11:20:54.464445: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla T4, Compute Capability 7.5\n",
      "2025-12-19 11:20:54.464777: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:20:54.466656: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1674] Found device 0 with properties: \n",
      "name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59\n",
      "pciBusID: 0000:00:1e.0\n",
      "2025-12-19 11:20:54.466725: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12\n",
      "2025-12-19 11:20:54.466900: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcublas.so.12\n",
      "2025-12-19 11:20:54.469118: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcufft.so.11\n",
      "2025-12-19 11:20:54.469223: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcurand.so.10\n",
      "2025-12-19 11:20:54.472814: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusolver.so.11\n",
      "2025-12-19 11:20:54.474022: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusparse.so.12\n",
      "2025-12-19 11:20:54.474128: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudnn.so.8\n",
      "2025-12-19 11:20:54.474293: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:20:54.476275: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:20:54.478112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1802] Adding visible gpu devices: 0\n",
      "2025-12-19 11:20:54.478160: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12\n",
      "2025-12-19 11:20:54.487015: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1214] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
      "2025-12-19 11:20:54.487061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1220]      0 \n",
      "2025-12-19 11:20:54.487075: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1233] 0:   N \n",
      "2025-12-19 11:20:54.487354: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:20:54.489360: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1082] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
      "2025-12-19 11:20:54.491209: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1359] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 13496 MB memory) -> physical GPU (device: 0, name: Tesla T4, pci bus id: 0000:00:1e.0, compute capability: 7.5)\n",
      "Telemetry data couldn't be sent, but the command ran successfully.\n",
      "[WARNING]: <urlopen error Url for the certificates not found.>\n",
      "Execution status: PASS\n",
      "2025-12-19 11:21:46,903 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 363: Stopping container.\n"
     ]
    }
   ],
   "source": [
    "# 4.1\n",
    "# DO NOT CHANGE THIS CELL\n",
    "# evaluate the model using the same validation set as training\n",
    "!tao model classification_tf1 evaluate -e $TAO_SPECS_DIR/vgg19/config.txt\\\n",
    "                                       -k $KEY \\\n",
    "                                       --log_file $TAO_PROJECT_DIR/log_file.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5384a9ac-c05b-4179-940a-9fdfab23337e",
   "metadata": {},
   "source": [
    "Click [here](tao_project/log_file.txt) to view the log file. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "7a796eaf-3086-48fa-b18c-7d227ad410c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 4.2\n",
    "# DO NOT CHANGE THIS CELL\n",
    "!cp $LOCAL_PROJECT_DIR/log_file.txt my_assessment/log_file.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aab7921b-734c-4cb1-8b91-7ab1b00892cb",
   "metadata": {},
   "source": [
    "### Grade Your Code ###\n",
    "If you have trained the model and completed model evaluation successfully, save changes to the notebook and revisit the webpage where you launched this interactive environment. Click on the \"**ASSESS TASK**\" button as shown in the screenshot below. Doing so will give you credit for this part of the lab that counts towards earning a certificate of competency for the entire course.\n",
    "<p><img src='images/credit.png' width=720></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9e13c81-abd4-43a2-bfe8-774116b3257f",
   "metadata": {},
   "source": [
    "<a href=\"https://www.nvidia.com/dli\"><img src=\"images/DLI_Header.png\" alt=\"Header\" style=\"width: 400px;\"/></a>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}