DTAT: backend.core (#301)

* Add files via upload

* cleared cell output

Author: Pwhsky

tutorials/3-advanced-topics/DTAT399A_backend.core.ipynb

@@ -0,0 +1,308 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# deeptrack.backend.core\n",
+    "\n",
+    "<a href=\"https://colab.research.google.com/github/DeepTrackAI/DeepTrack2/blob/develop/tutorials/3-advanced-topics/DTAT399A_backend.core.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# !pip install deeptrack  # Uncomment if running on Colab/Kaggle."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This advanced tutorial introduces the backend.core module."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. What is `core`?\n",
+    "\n",
+    "The `core` module provides fundamental utilities and functions to manage and process data on a low level.\n",
+    "\n",
+    "In particular it provide tools to store, validate, and manage data and computational nodes with dependency tracking.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Basic Node Usage with Parent-Child Dependency"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "30 40\n",
+      "False\n",
+      "False\n",
+      "50\n"
+     ]
+    }
+   ],
+   "source": [
+    "from deeptrack.backend.core import DeepTrackNode\n",
+    "\n",
+    "parent = DeepTrackNode(action=lambda: 10)\n",
+    "child = DeepTrackNode(action=lambda _ID=None: parent(_ID) * 2)\n",
+    "\n",
+    "# Establish parent-child dependency.\n",
+    "parent.add_child(child)\n",
+    "\n",
+    "# Store values.\n",
+    "parent.store(15, _ID=(0,))\n",
+    "parent.store(20, _ID=(1,))\n",
+    "\n",
+    "# Compute values based on parent values.\n",
+    "child_value_0 = child(_ID=(0,))\n",
+    "child_value_1 = child(_ID=(1,))\n",
+    "print(child_value_0, child_value_1)\n",
+    "\n",
+    "# Invalidate parent data for a given ID.\n",
+    "parent.invalidate((0,))\n",
+    "print(parent.is_valid((0,)))\n",
+    "\n",
+    "# Update the parent value and recompute the child value:\n",
+    "print(child.is_valid((0,)))\n",
+    "parent.store(25, _ID=(0,))\n",
+    "child_value_recomputed = child(_ID=(0,))\n",
+    "print(child_value_recomputed)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3.  Lazy evaluation and Caching\n",
+    "Here we add a function to a `DeepTrackNode` which retuns a constant value and updates a global counter variable when called."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 95,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "10 1\n",
+      "10 2\n",
+      "10 3\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Create counter node with side effect\n",
+    "call_count = 0\n",
+    "def calculation():\n",
+    "    global call_count\n",
+    "    call_count += 1\n",
+    "    return 10\n",
+    "\n",
+    "node = DeepTrackNode(calculation)\n",
+    "\n",
+    "# First call computes value.\n",
+    "print(node(), call_count) \n",
+    "\n",
+    "# Subsequent call uses cached value.\n",
+    "node.invalidate()\n",
+    "print(node(), call_count) \n",
+    "\n",
+    "# Invalidate and call again.\n",
+    "node.invalidate()\n",
+    "print(node(), call_count) "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Data Management with IDs\n",
+    "\n",
+    "Map IDs to stored `DeepTrackData` objects lika a dictionary."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Cat\n",
+      "Bird\n",
+      "{(0, 0): <deeptrack.backend.core.DeepTrackDataObject object at 0x7f0dc3022b90>, (0, 1): <deeptrack.backend.core.DeepTrackDataObject object at 0x7f0dc302f790>}\n"
+     ]
+    }
+   ],
+   "source": [
+    "from deeptrack.backend.core import DeepTrackDataDict\n",
+    "\n",
+    "data_dict = DeepTrackDataDict()\n",
+    "\n",
+    "# Create listings with unique indices.\n",
+    "data_dict.create_index((0, 0))\n",
+    "data_dict.create_index((0, 1))\n",
+    "data_dict.create_index((1, 0))\n",
+    "data_dict.create_index((1, 1))\n",
+    "\n",
+    "# Store some data for the indices.\n",
+    "data_dict[(0, 0)].store(\"Cat\")\n",
+    "data_dict[(0, 1)].store(\"Dog\")\n",
+    "data_dict[(1, 0)].store(\"Mouse\")\n",
+    "data_dict[(1, 1)].store(\"Bird\")\n",
+    "\n",
+    "# Print the indices.\n",
+    "print(data_dict[(0, 0)].current_value())\n",
+    "print(data_dict[(1, 1)].current_value())\n",
+    "print(data_dict[(0, )])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Propagating operators\n",
+    "Nodes can also be used as simple handles for functions."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 92,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "16\n",
+      "60\n"
+     ]
+    }
+   ],
+   "source": [
+    "a = DeepTrackNode(lambda: 5 + 5)\n",
+    "b = DeepTrackNode(lambda: 3 + 3)\n",
+    "\n",
+    "sum_node = a + b\n",
+    "product_node = a * b\n",
+    "\n",
+    "print(sum_node())\n",
+    "print(product_node())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Validation control\n",
+    "Validate or invalidate nodes manually to enable/disable storing data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 88,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "100\n",
+      "True\n",
+      "100\n",
+      "False\n",
+      "42\n"
+     ]
+    }
+   ],
+   "source": [
+    "node = DeepTrackNode(lambda: 42)\n",
+    "node.store(100)\n",
+    "\n",
+    "print(node())\n",
+    "\n",
+    "# Validate.\n",
+    "node.validate()\n",
+    "print(node.is_valid())\n",
+    "print(node()) \n",
+    "\n",
+    "# Invalidate.\n",
+    "node.invalidate()\n",
+    "print(node.is_valid())\n",
+    "print(node())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 7. Get Citations\n",
+    "The `DeepTrackNode` class can also be used to obtain citations."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 101,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'\\n@article{Midtvet2021DeepTrack,\\n    author  = {Midtvedt,Benjamin  and \\n               Helgadottir,Saga  and \\n               Argun,Aykut  and \\n               Pineda,Jesús  and \\n               Midtvedt,Daniel  and \\n               Volpe,Giovanni},\\n    title   = {Quantitative digital microscopy with deep learning},\\n    journal = {Applied Physics Reviews},\\n    volume  = {8},\\n    number  = {1},\\n    pages   = {011310},\\n    year    = {2021},\\n    doi     = {10.1063/5.0034891}\\n}\\n'}"
+      ]
+     },
+     "execution_count": 101,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "DeepTrackNode().get_citations()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}