lab-05-1-logistic_regression-eager.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 05 Logistic Classification(Regression) - Eager Execution\n",
    "* Logistic Classfication은 True or False와 같은 Binary나 복수개의 다항 분류에 쓰입니다 (Bernoulli Distribution)\n",
    "\n",
    "### 기본 Library 선언 및 Tensorflow 버전 확인"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.12.0\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import tensorflow as tf\n",
    "import tensorflow.contrib.eager as tfe\n",
    "\n",
    "tf.enable_eager_execution()\n",
    "tf.set_random_seed(777)  # for reproducibility\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 강의에 설명할 Data입니다\n",
    "* x_data가 2차원 배열이기에 2차원 공간에 표현하여 x1과 x2를 기준으로 y_data 0과 1로 구분하는 예제입니다\n",
    "* Logistic Classification 통해 보라색과 노란색 y_data(Label)을 구분해 보겠습니다.\n",
    "* Test 데이터는 붉은색의 위치와 같이 추론시 1의 값을 가지게 됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFwhJREFUeJzt3X+w3XWd3/HnKz8UCKy/uLIWiLEt265aAb2iDlTBrQhWS+04LYxFx2Iz47hd2Tq2Cl2oWHeq7tLudlQahaIYUFdA2R1AYmWLyIDcsOE3rgygJEObIAgJgfx894/zvXoIN8kncL853Hufj5kz55zP9/P9ft9nCHnl8/l+z/mkqpAkaXfmjboASdLMYGBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWqyYNQFTKcDDzywlixZMuoyJGnGWLly5cNVNdbSd1YFxpIlS5iYmBh1GZI0YyT5eWtfp6QkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUpPeAiPJPkl+kuTWJHcm+fQUfV6Y5FtJ7k1yU5IlQ9s+1bX/NMk7+6pzrlr+2Uv51ue/O+oy1KPacjvbHzmNqu2jLmWv2b7+v7H9ia+PuoxZq8/vYWwC3l5VG5IsBK5PclVV3TjU5zTg0ar6+0lOBj4H/KskrwZOBl4D/B3gB0l+p6q29VjvnPHYw49z8R9fxryEd/3bf8IBL9l/1CWpB/X4H8OWv4FNP4B9jh91Ob2rbWvhiQsg86l9/wWZ55/r6dbbCKMGNnRvF3aPHRcQPwn4Wvf6O8DvJUnX/s2q2lRV9wP3Akf1Vetcc8l/vRyq2L59O3/xJ1eMuhz1oDavgi13Atup9Z+fE6OM2vBFYDvUdmrj13bbX3uu12sYSeYnWQWsBVZU1U07dDkYeBCgqrYCjwEvG27vrO7a9Bw99vDj/OWXr2HzU1vY/NQWLv+zK1n/6Ibd76gZpdZ/jsEgH9j28GCUMYvVtrXw5GXAFuApeOIr1Hb/XE+3XgOjqrZV1RHAIcBRSV473edIsjTJRJKJdevWTffhZ53J0cUkRxmzz29GF5P/nTfO+lHGr0cXv25wlNGHvXKXVFX9CrgWOGGHTWuAQwGSLABeBPxyuL1zSNc21bGXVdV4VY2PjTX9ftacNTy6mOQoY/Z52uhi0iweZTx9dDHJUUYf+rxLaizJi7vX+wLvAO7ZodsVwAe71+8DflhV1bWf3N1F9SrgMOAnfdU6V1x/2U1seWoLL9j3BU97bHpyMz/+7s2jLk/ToLatgy0rGVwyfOHQYxO18Zsjra03T10NbObpn/eFUBth01+PsrJZJ1U7XoeepgMnr2NwQXs+g2D6dlWdk+QcYKKqrkiyD3ARcCTwCHByVd3X7X8m8G+ArcDpVXXV7s45Pj5e/lrtzm3bto3HH14/5bYXjf0W8+b5tZzZoLb/CmrrMzfM25/B/3KzS9U22P7o1BvnvYzBfTTamSQrq2q8qW9fgTEKBoYk7Zk9CQz/SSlJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYL+jpwkkOBrwMHMVhceFlV/dkOfT4BvH+olt8FxqrqkSQPAOuBbcDW1t9rlyT1o7fAYLBS3ser6pYkBwArk6yoqrsmO1TVF4AvACR5D/CHVfXI0DGOq6qHe6xRktSotympqnqoqm7pXq8H7gYO3sUupwCX9FWPJOm52SvXMJIsYbBu90072b4fcAJw6VBzAdckWZlkad81SpJ2rc8pKQCS7M8gCE6vqsd30u09wI93mI46pqrWJHk5sCLJPVV13RTHXwosBVi8ePE0Vy9JmtTrCCPJQgZhsbyqLttF15PZYTqqqtZ0z2uBy4GjptqxqpZV1XhVjY+NjU1P4ZKkZ+gtMJIEOB+4u6rO3UW/FwFvA7431Laou1BOkkXA8cAdfdUqSdq9PqekjgZOBW5PsqprOwNYDFBV53Vt7wWuqaonhvY9CLh8kDksAC6uqqt7rFWStBu9BUZVXQ+kod+FwIU7tN0HHN5LYZKkZ8VvekuSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlq0ucSrYcmuTbJXUnuTPKxKfocm+SxJKu6x1lD205I8tMk9yb5ZF91SpLa9LlE61bg41V1S7c+98okK6rqrh36/aiq3j3ckGQ+8EXgHcBq4OYkV0yxryRpL+lthFFVD1XVLd3r9cDdwMGNux8F3FtV91XVZuCbwEn9VCpJarFXrmEkWQIcCdw0xea3JLk1yVVJXtO1HQw8ONRnNTsJmyRLk0wkmVi3bt00Vi1JGtZ7YCTZH7gUOL2qHt9h8y3AK6vqcOB/AN/d0+NX1bKqGq+q8bGxsedesCRpSr0GRpKFDMJieVVdtuP2qnq8qjZ0r68EFiY5EFgDHDrU9ZCuTZI0In3eJRXgfODuqjp3J31+u+tHkqO6en4J3AwcluRVSV4AnAxc0VetkqTd6/MuqaOBU4Hbk6zq2s4AFgNU1XnA+4CPJNkKPAmcXFUFbE3y+8D3gfnABVV1Z4+1SpJ2I4O/n2eH8fHxmpiYGHUZkjRjJFlZVeMtff2mtySpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmvS54t6hSa5NcleSO5N8bIo+709yW5Lbk9yQ5PChbQ907auSuMiFJI1YnyvubQU+XlW3JDkAWJlkRVXdNdTnfuBtVfVokhOBZcCbhrYfV1UP91ijJKlRb4FRVQ8BD3Wv1ye5GzgYuGuozw1Du9wIHNJXPZKk52avXMNIsgQ4ErhpF91OA64ael/ANUlWJlnaX3WSpBZ9TkkBkGR/4FLg9Kp6fCd9jmMQGMcMNR9TVWuSvBxYkeSeqrpuin2XAksBFi9ePO31S5IGeh1hJFnIICyWV9VlO+nzOuCrwElV9cvJ9qpa0z2vBS4Hjppq/6paVlXjVTU+NjY23R9BktTp8y6pAOcDd1fVuTvpsxi4DDi1qv52qH1Rd6GcJIuA44E7+qpVkrR7fU5JHQ2cCtyeZFXXdgawGKCqzgPOAl4GfGmQL2ytqnHgIODyrm0BcHFVXd1jrZKk3ejzLqnrgeymz4eBD0/Rfh9w+DP3kCSNit/0liQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSkz5X3Ds0ybVJ7kpyZ5KPTdEnSf48yb1Jbkvy+qFtH0zys+7xwb7qBNi8aQu3XXdXn6eQpF7Uppuo2rpXzrXLwEjyW0n+3hTtr2s49lbg41X1auDNwEeTvHqHPicCh3WPpcCXu+O/FDgbeBODtbzPTvKShnM+K3/5pav5xNv/Mw/d///6OoWkvi1fDkuWwLx5g+fly0ddUe9qy8+oR0+lnvzuXjnfTgMjyb8E7gEu7UYIbxzafOHuDlxVD1XVLd3r9cDdwME7dDsJ+HoN3Ai8OMkrgHcCK6rqkap6FFgBnLAHn6vZpic3cdE534GEC8/6Vh+nkNS35cth6VL4+c+havC8dOmsD43a8CdAYMO5VG3p/Xy7GmGcAbyhqo4APgRclOS93bZdLr26oyRLgCOBm3bYdDDw4ND71V3bztqn3V+ddw3btm5j+7btXH/pjY4ypJnozDNh48ant23cOGifpWrLz2DTDUBBbaSe/F7v59xVYMyvqocAquonwHHAf0ryB4MK2yTZH7gUOL2qHn8uxe7k+EuTTCSZWLdu3R7tOzm6eOqJTQBs27rdUYY0E/3iF3vWPgsMRhfdqKI27pVRxq4CY/3w9YsuPI5lMI30mpaDJ1nIICyWV9VlU3RZAxw69P6Qrm1n7c9QVcuqaryqxsfGxlrK+rXJ0cWkbVu3OcqQZqLFi/esfYb7zehi+1Bj/6OMXQXGR4B5wxequ2sRJwAf3t2BkwQ4H7i7qs7dSbcrgA90d0u9GXisC6bvA8cneUl3sfv4rm3abH5q89NGF5O2bN7K187+9nSeSlLfPvtZ2G+/p7ftt9+gfRaqDX8KbN6hcXKUsW3KfabDgp0WVHUrQJI7klwEfB7Yp3seBy7azbGPBk4Fbk+yqms7A1jcHf884ErgXcC9wEYG10qoqkeSfAa4udvvnKp6ZI8/3S5s31684wNve0ZgAPzOG/7udJ5KUt/e//7B85lnDqahFi8ehMVk+2zzgjfBvJc+sz2LGIw65vdy2lTt+nJEkkXA54A3AAcAy4HPVdX2Xe44AuPj4zUxMTHqMiRpxkiysqrGW/q2fHFvC/AksC+DEcb9z8ewkCT1qyUwbmYQGG8E/jFwSpK/6LUqSdLzzk6vYQw5raom53keAk5KcmqPNUmSnod2O8IYCovhtt1d8JYkzTL+Wq0kqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmLb9W+6wkuQB4N7C2ql47xfZPAJPLYS0AfhcY61bbewBYD2wDtrYu7iFJ6k+fI4wLGaz/PaWq+kJVHVFVRwCfAv7PDsuwHtdtNywk6Xmgt8CoquuA1nW4TwEu6asWSdJzN/JrGEn2YzASuXSouYBrkqxMsnQ3+y9NMpFkYt26dX2WKklz2sgDA3gP8OMdpqOOqarXAycCH03y1p3tXFXLqmq8qsbHxsb6rlWS5qznQ2CczA7TUVW1pnteC1wOHDWCuiRJQ0YaGEleBLwN+N5Q26IkB0y+Bo4H7hhNhZKkSX3eVnsJcCxwYJLVwNnAQoCqOq/r9l7gmqp6YmjXg4DLk0zWd3FVXd1XnZKkNr0FRlWd0tDnQga33w633Qcc3k9VkqRn6/lwDUOSNAMYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKlJb4GR5IIka5NMuVpekmOTPJZkVfc4a2jbCUl+muTeJJ/sq0ZJUrs+RxgXAifsps+PquqI7nEOQJL5wBeBE4FXA6ckeXWPdUqSGvQWGFV1HfDIs9j1KODeqrqvqjYD3wROmtbiJEl7bNTXMN6S5NYkVyV5Tdd2MPDgUJ/VXZskaYR6W9O7wS3AK6tqQ5J3Ad8FDtvTgyRZCiwFWLx48fRWKEn6tZGNMKrq8ara0L2+EliY5EBgDXDoUNdDuradHWdZVY1X1fjY2FivNUvSXDaywEjy20nSvT6qq+WXwM3AYUleleQFwMnAFaOqU5I00NuUVJJLgGOBA5OsBs4GFgJU1XnA+4CPJNkKPAmcXFUFbE3y+8D3gfnABVV1Z191SpLaZPB39OwwPj5eExMToy5DkmaMJCuraryl76jvkpIkzRAGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmvQWGEkuSLI2yR072f7+JLcluT3JDUkOH9r2QNe+KokrIknS80CfI4wLgRN2sf1+4G1V9Y+AzwDLdth+XFUd0boSlCSpX72t6V1V1yVZsovtNwy9vRE4pK9aJEnP3fPlGsZpwFVD7wu4JsnKJEt3tWOSpUkmkkysW7eu1yIlaS7rbYTRKslxDALjmKHmY6pqTZKXAyuS3FNV1021f1Uto5vOGh8fr94LlqQ5aqQjjCSvA74KnFRVv5xsr6o13fNa4HLgqNFUKEmaNLLASLIYuAw4tar+dqh9UZIDJl8DxwNT3mklSdp7epuSSnIJcCxwYJLVwNnAQoCqOg84C3gZ8KUkAFu7O6IOAi7v2hYAF1fV1X3VKUlq0+ddUqfsZvuHgQ9P0X4fcPgz95AkjdLz5S4pSdLznIEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmvQaGEkuSLI2yZQr5mXgz5Pcm+S2JK8f2vbBJD/rHh/ss07Nflu3bOUP3/pH3H/HL0ZdijRj9T3CuBA4YRfbTwQO6x5LgS8DJHkpgxX63sRgPe+zk7yk10o1q/3gouu484af8pX/+I1RlyLNWL0GRlVdBzyyiy4nAV+vgRuBFyd5BfBOYEVVPVJVjwIr2HXwSDu1dctWLjjzYmp7cdtf38n9t/981CVJM9Kor2EcDDw49H5117azdmmP/eCi63jyiU0AbN60ha98cvmIK5JmplEHxnOWZGmSiSQT69atG3U5ep6ZHF08teEpAEcZ0nMw6sBYAxw69P6Qrm1n7c9QVcuqaryqxsfGxnorVDPT8OhikqMM6dkZdWBcAXygu1vqzcBjVfUQ8H3g+CQv6S52H9+1SXvk6gt+yOYnN7PwhQt//Zg/fx4rr7mVJx7fOOrypBllQZ8HT3IJcCxwYJLVDO58WghQVecBVwLvAu4FNgIf6rY9kuQzwM3doc6pql1dPJem9PkfnMVTGzc9o33BwgXsd8C+I6hImrlSVaOuYdqMj4/XxMTEqMuQpBkjycqqGm/pO+opKUnSDGFgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmsyq22qTrAOe7W8+HAg8PI3lzAR+5tlvrn1e8DPvqVdWVdPPZMyqwHgukky03os8W/iZZ7+59nnBz9wnp6QkSU0MDElSEwPjN5aNuoAR8DPPfnPt84KfuTdew5AkNXGEIUlqMucDI8kFSdYmuWPUtewtSQ5Ncm2Su5LcmeRjo66pT0n2SfKTJLd2n/fTo65pb0kyP8nfJPmrUdeyNyR5IMntSVYlmfU/XZ3kxUm+k+SeJHcneUuv55vrU1JJ3gpsAL5eVa8ddT17Q5JXAK+oqluSHACsBP55Vd014tJ6kSTAoqrakGQhcD3wsaq6ccSl9S7JvwfGgd+qqnePup6+JXkAGK+qOfE9jCRfA35UVV9N8gJgv6r6VV/nm/MjjKq6DphTizNV1UNVdUv3ej1wN3DwaKvqTw1s6N4u7B6z/l9KSQ4B/inw1VHXoumX5EXAW4HzAapqc59hAQbGnJdkCXAkcNNoK+lXNzWzClgLrKiqWf15O/8d+A/A9lEXshcVcE2SlUmWjrqYnr0KWAf8r27a8atJFvV5QgNjDkuyP3ApcHpVPT7qevpUVduq6gjgEOCoJLN6+jHJu4G1VbVy1LXsZcdU1euBE4GPdlPOs9UC4PXAl6vqSOAJ4JN9ntDAmKO6ufxLgeVVddmo69lbuiH7tcAJo66lZ0cD/6yb0/8m8PYk3xhtSf2rqjXd81rgcuCo0VbUq9XA6qHR8ncYBEhvDIw5qLsIfD5wd1WdO+p6+pZkLMmLu9f7Au8A7hltVf2qqk9V1SFVtQQ4GfhhVf3rEZfVqySLups46KZmjgdm7d2PVfV/gQeT/IOu6feAXm9cWdDnwWeCJJcAxwIHJlkNnF1V54+2qt4dDZwK3N7N6wOcUVVXjrCmPr0C+FqS+Qz+kfTtqpoTt5nOMQcBlw/+PcQC4OKqunq0JfXu3wHLuzuk7gM+1OfJ5vxttZKkNk5JSZKaGBiSpCYGhiSpiYEhSWpiYEiSmhgY0l6Q5Ookv5orvxqr2cnAkPaOLzD47os0YxkY0jRK8sYkt3VrcCzq1t94bVX9b2D9qOuTnos5/01vaTpV1c1JrgD+C7Av8I2qmrU/T6G5xcCQpt85wM3AU8AfjLgWado4JSVNv5cB+wMHAPuMuBZp2hgY0vT7n8AfAcuBz424FmnaOCUlTaMkHwC2VNXF3a/j3pDk7cCngX8I7N/9KvJpVfX9UdYq7Sl/rVaS1MQpKUlSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTf4/hd6eFZKLVBYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_train = [[1., 2.],\n",
    "          [2., 3.],\n",
    "          [3., 1.],\n",
    "          [4., 3.],\n",
    "          [5., 3.],\n",
    "          [6., 2.]]\n",
    "y_train = [[0.],\n",
    "          [0.],\n",
    "          [0.],\n",
    "          [1.],\n",
    "          [1.],\n",
    "          [1.]]\n",
    "\n",
    "x_test = [[5.,2.]]\n",
    "y_test = [[1.]]\n",
    "\n",
    "\n",
    "x1 = [x[0] for x in x_train]\n",
    "x2 = [x[1] for x in x_train]\n",
    "\n",
    "colors = [int(y[0] % 3) for y in y_train]\n",
    "plt.scatter(x1,x2, c=colors , marker='^')\n",
    "plt.scatter(x_test[0][0],x_test[0][1], c=\"red\")\n",
    "\n",
    "plt.xlabel(\"x1\")\n",
    "plt.ylabel(\"x2\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Tensorflow Eager\n",
    "### 위 Data를 기준으로 가설의 검증을 통해 Logistic Classification 모델을 만들도록 하겠습니다\n",
    "* Tensorflow data API를 통해 학습시킬 값들을 담는다 (Batch Size는 한번에 학습시킬 Size로 정한다)\n",
    "* features,labels는 실재 학습에 쓰일 Data (연산을 위해 Type를 맞춰준다)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(len(x_train))#.repeat()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 위 Data를 기준으로 가설의 검증을 통해 Logistic Classification 모델을 만들도록 하겠습니다\n",
    "* W와 b은 학습을 통해 생성되는 모델에 쓰이는 Wegith와 Bias (초기값을 variable : 0이나 Random값으로 가능 tf.random_normal([2, 1]) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "W = tf.Variable(tf.zeros([2,1]), name='weight')\n",
    "b = tf.Variable(tf.zeros([1]), name='bias')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sigmoid 함수를 가설로 선언합니다\n",
    "* Sigmoid는 아래 그래프와 같이 0과 1의 값만을 리턴합니다 tf.sigmoid(tf.matmul(X, W) + b)와 같습니다\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "sigmoid(x) & = \\frac{1}{1+e^{-x}}  \\\\\\\\\\\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "![sigmoid](https://upload.wikimedia.org/wikipedia/commons/8/88/Logistic-curve.svg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def logistic_regression(features):\n",
    "    hypothesis  = tf.div(1., 1. + tf.exp(tf.matmul(features, W) + b))\n",
    "    return hypothesis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 가설을 검증할 Cost 함수를 정의합니다\n",
    "$$\n",
    "\\begin{align}\n",
    "cost(h(x),y) & = −log(h(x))  &  if  &  y=1 \\\\\\\\\\\n",
    "cost(h(x),y) & = -log(1−h(x))  &  if  &  y=0\n",
    "\\end{align}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 위 두수식을 합치면 아래과 같습니다\n",
    "$$\n",
    "\\begin{align}\n",
    "cost(h(x),y) & = −y log(h(x))−(1−y)log(1−h(x))\n",
    "\\end{align}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_fn(hypothesis, features, labels):\n",
    "    cost = -tf.reduce_mean(labels * tf.log(logistic_regression(features)) + (1 - labels) * tf.log(1 - hypothesis))\n",
    "    return cost\n",
    "\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 추론한 값은 0.5를 기준(Sigmoid 그래프 참조)로 0과 1의 값을 리턴합니다.\n",
    "* Sigmoid 함수를 통해 예측값이 0.5보다 크면 1을 반환하고 0.5보다 작으면 0으로 반환합니다.\n",
    "* 가설을 통해 실재 값과 비교한 정확도를 측정합니다"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def accuracy_fn(hypothesis, labels):\n",
    "    predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)\n",
    "    accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, labels), dtype=tf.int32))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GradientTape를 통해 경사값을 계산합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def grad(hypothesis, features, labels):\n",
    "    with tf.GradientTape() as tape:\n",
    "        loss_value = loss_fn(logistic_regression(features),features,labels)\n",
    "    return tape.gradient(loss_value, [W,b])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Eager모드에서 학습을 실행합니다.\n",
    "* 위의 Data를 Cost함수를 통해 학습시킨 후 모델을 생성합니다. \n",
    "* 새로운 Data를 통한 검증 수행 [5,2]의 Data로 테스트 수행 (그래프상 1이 나와야 정상입니다)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter: 0, Loss: 0.6874\n",
      "Iter: 100, Loss: 0.5776\n",
      "Iter: 200, Loss: 0.5349\n",
      "Iter: 300, Loss: 0.5054\n",
      "Iter: 400, Loss: 0.4838\n",
      "Iter: 500, Loss: 0.4671\n",
      "Iter: 600, Loss: 0.4535\n",
      "Iter: 700, Loss: 0.4420\n",
      "Iter: 800, Loss: 0.4319\n",
      "Iter: 900, Loss: 0.4228\n",
      "Iter: 1000, Loss: 0.4144\n",
      "Testset Accuracy: 1.0000\n"
     ]
    }
   ],
   "source": [
    "EPOCHS = 1001\n",
    "\n",
    "for step in range(EPOCHS):\n",
    "    for features, labels  in tfe.Iterator(dataset):\n",
    "        grads = grad(logistic_regression(features), features, labels)\n",
    "        optimizer.apply_gradients(grads_and_vars=zip(grads,[W,b]))\n",
    "        if step % 100 == 0:\n",
    "            print(\"Iter: {}, Loss: {:.4f}\".format(step, loss_fn(logistic_regression(features),features,labels)))\n",
    "test_acc = accuracy_fn(logistic_regression(x_test),y_test)\n",
    "print(\"Testset Accuracy: {:.4f}\".format(test_acc))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}