{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.13.0"
    },
    "blog_metadata": {
      "topic": "GPT-5.6 on Azure Databricks: What Frontier Models Mean for Data Platform Economics",
      "slug": "gpt-5-6-on-azure-databricks-what-frontier-models-mean-for-da",
      "generated_by": "LinkedIn Post Generator + Azure OpenAI",
      "generated_at": "2026-07-10T13:37:45.465Z"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# GPT-5.6 on Azure Databricks: What Frontier Models Mean for Data Platform Economics\n",
        "\n",
        "This notebook turns the blog post into a hands-on validation flow focused on inference placement, governance, and chargeback on Azure Databricks. The goal is not just to call a frontier model endpoint, but to make usage legible through metadata capture, lightweight persistence, and operational inspection patterns."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "%pip install -q requests pyspark"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "import os\n",
        "import time\n",
        "import json\n",
        "import requests\n",
        "from datetime import datetime\n",
        "from pyspark.sql import Row\n",
        "from pyspark.sql import functions as F"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Platform economics framing\n",
        "\n",
        "The central design question is no longer only which model to use, but where inference should live. In Azure Databricks, that choice affects latency, concurrency, governance, ownership, and chargeback.\n",
        "\n",
        "Typical workload classes to validate:\n",
        "- Interactive copilots: optimize for latency, concurrency, and guardrails\n",
        "- Batch enrichment: optimize for throughput and repeatability\n",
        "- Evaluation runs: optimize for observability and spend control\n",
        "- Production APIs: optimize for quotas, ownership, and predictable behavior\n",
        "\n",
        "A practical principle from the post: the cheapest model is often not the cheapest production path. The better path is usually the one with the lowest governance friction and clearest cost attribution."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Required environment variables\n",
        "\n",
        "Set these before running the API validation cells:\n",
        "- `DATABRICKS_HOST`: Databricks workspace URL, for example `https://adb-<workspace>.azuredatabricks.net`\n",
        "- `DATABRICKS_TOKEN`: Personal access token or equivalent auth token with permission to invoke serving endpoints\n",
        "\n",
        "Optional assumptions used in examples:\n",
        "- Endpoint name: `databricks-gpt-5-6`\n",
        "- Target table: `finops.model_usage_events`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Example 1: Minimal model invocation with usage metadata\n",
        "\n",
        "This example validates the minimum viable chargeback pattern described in the post. It calls a Databricks serving endpoint, measures latency, extracts token usage if returned by the API, and stamps the event with workload class and cost center so the result can be routed into a shared FinOps view."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "host = os.environ.get(\"DATABRICKS_HOST\")\n",
        "token = os.environ.get(\"DATABRICKS_TOKEN\")\n",
        "endpoint = \"databricks-gpt-5-6\"\n",
        "\n",
        "if not host or not token:\n",
        "    raise EnvironmentError(\"Missing required environment variables: DATABRICKS_HOST and/or DATABRICKS_TOKEN\")\n",
        "\n",
        "url = f\"{host}/serving-endpoints/{endpoint}/invocations\"\n",
        "payload = {\n",
        "    \"messages\": [\n",
        "        {\"role\": \"user\", \"content\": \"Summarize yesterday's sales anomalies in 3 bullets.\"}\n",
        "    ]\n",
        "}\n",
        "headers = {\n",
        "    \"Authorization\": f\"Bearer {token}\",\n",
        "    \"Content-Type\": \"application/json\"\n",
        "}\n",
        "\n",
        "t0 = time.time()\n",
        "r = requests.post(url, headers=headers, json=payload, timeout=60)\n",
        "latency_ms = int((time.time() - t0) * 1000)\n",
        "\n",
        "try:\n",
        "    response_json = r.json()\n",
        "except ValueError:\n",
        "    response_json = {\"raw_text\": r.text}\n",
        "\n",
        "usage = (response_json.get(\"usage\") or {}) if isinstance(response_json, dict) else {}\n",
        "event = {\n",
        "    \"endpoint\": endpoint,\n",
        "    \"workload_class\": \"bi-copilot\",\n",
        "    \"cost_center\": \"finance-analytics\",\n",
        "    \"latency_ms\": latency_ms,\n",
        "    \"http_status\": r.status_code,\n",
        "    \"prompt_tokens\": usage.get(\"prompt_tokens\"),\n",
        "    \"completion_tokens\": usage.get(\"completion_tokens\"),\n",
        "    \"total_tokens\": usage.get(\"total_tokens\")\n",
        "}\n",
        "\n",
        "print(\"Response preview:\")\n",
        "print(json.dumps(response_json, indent=2)[:4000])\n",
        "print(\"\\nUsage event:\")\n",
        "print(json.dumps(event, indent=2))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Example 2: Persist a lightweight governance record\n",
        "\n",
        "The model response is only part of the story. The more useful artifact is the usage event, because it enables shared visibility across platform engineering, data leadership, and FinOps.\n",
        "\n",
        "This example writes a lightweight governance record to a Delta-backed table so teams can compare workload classes, latency, and token consumption over time."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "if \"event\" not in globals():\n",
        "    event = {\n",
        "        \"endpoint\": \"databricks-gpt-5-6\",\n",
        "        \"workload_class\": \"bi-copilot\",\n",
        "        \"cost_center\": \"finance-analytics\",\n",
        "        \"latency_ms\": 0,\n",
        "        \"total_tokens\": None\n",
        "    }\n",
        "\n",
        "spark.sql(\"CREATE DATABASE IF NOT EXISTS finops\")\n",
        "\n",
        "record = Row(\n",
        "    ts_utc=datetime.utcnow().isoformat(),\n",
        "    endpoint=\"databricks-gpt-5-6\",\n",
        "    owner_team=\"data-platform\",\n",
        "    workload_class=event[\"workload_class\"],\n",
        "    environment=\"prod\",\n",
        "    total_tokens=event.get(\"total_tokens\"),\n",
        "    latency_ms=event.get(\"latency_ms\"),\n",
        "    cost_center=event.get(\"cost_center\")\n",
        ")\n",
        "\n",
        "spark.createDataFrame([record]).write.mode(\"append\").saveAsTable(\"finops.model_usage_events\")\n",
        "\n",
        "spark.table(\"finops.model_usage_events\").orderBy(F.col(\"ts_utc\").desc()).show(10, truncate=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Example 3: Inspect endpoint ownership and operational state\n",
        "\n",
        "The original post included a PowerShell example for inspecting a serving endpoint. Here the same validation is expressed in Python so it can run directly in this notebook.\n",
        "\n",
        "This helps answer operational questions such as who created the endpoint, whether it is ready, and what served entity is attached. Those details matter when deciding whether inference belongs to the data platform team, an app team, or a separate AI estate."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "host = os.environ.get(\"DATABRICKS_HOST\")\n",
        "token = os.environ.get(\"DATABRICKS_TOKEN\")\n",
        "endpoint = \"databricks-gpt-5-6\"\n",
        "\n",
        "if not host or not token:\n",
        "    raise EnvironmentError(\"Missing required environment variables: DATABRICKS_HOST and/or DATABRICKS_TOKEN\")\n",
        "\n",
        "uri = f\"{host}/api/2.0/serving-endpoints/{endpoint}\"\n",
        "headers = {\"Authorization\": f\"Bearer {token}\"}\n",
        "response = requests.get(uri, headers=headers, timeout=60)\n",
        "response.raise_for_status()\n",
        "endpoint_info = response.json()\n",
        "\n",
        "served_entities = ((endpoint_info.get(\"config\") or {}).get(\"served_entities\") or [])\n",
        "first_entity = served_entities[0] if served_entities else {}\n",
        "summary = {\n",
        "    \"name\": endpoint_info.get(\"name\"),\n",
        "    \"creator\": endpoint_info.get(\"creator\"),\n",
        "    \"state\": ((endpoint_info.get(\"state\") or {}).get(\"ready\")),\n",
        "    \"task\": first_entity.get(\"task\"),\n",
        "    \"entity_name\": first_entity.get(\"entity_name\")\n",
        "}\n",
        "\n",
        "print(json.dumps(summary, indent=2))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Example 4: Patch endpoint configuration to illustrate deployment controls\n",
        "\n",
        "The post argues that the hidden bill is not just model rates, but also the serving mode, duplicated controls, and weak ownership standards. This example shows how deployment controls can be updated through the endpoint configuration API.\n",
        "\n",
        "Use caution: this is a live configuration change. Review endpoint schema and permissions in your workspace before running it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "host = os.environ.get(\"DATABRICKS_HOST\")\n",
        "token = os.environ.get(\"DATABRICKS_TOKEN\")\n",
        "endpoint = \"databricks-gpt-5-6\"\n",
        "\n",
        "if not host or not token:\n",
        "    raise EnvironmentError(\"Missing required environment variables: DATABRICKS_HOST and/or DATABRICKS_TOKEN\")\n",
        "\n",
        "uri = f\"{host}/api/2.0/serving-endpoints/{endpoint}/config\"\n",
        "headers = {\n",
        "    \"Authorization\": f\"Bearer {token}\",\n",
        "    \"Content-Type\": \"application/json\"\n",
        "}\n",
        "body = {\n",
        "    \"served_entities\": [\n",
        "        {\n",
        "            \"name\": \"gpt56-prod\",\n",
        "            \"scale_to_zero_enabled\": True,\n",
        "            \"workload_size\": \"Small\",\n",
        "            \"environment_vars\": {\n",
        "                \"COST_CENTER\": \"finance-analytics\",\n",
        "                \"OWNER\": \"data-platform\"\n",
        "            }\n",
        "        }\n",
        "    ]\n",
        "}\n",
        "\n",
        "print(\"About to submit config payload:\")\n",
        "print(json.dumps(body, indent=2))\n",
        "\n",
        "# Uncomment to apply the change in a real environment.\n",
        "# patch_response = requests.put(uri, headers=headers, json=body, timeout=60)\n",
        "# patch_response.raise_for_status()\n",
        "# print(json.dumps(patch_response.json(), indent=2))\n",
        "\n",
        "print(\"Dry run only. Remove comments around the requests.put call to execute.\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Architecture flow for chargeback and governance\n",
        "\n",
        "The blog's architecture can be represented as a simple flow:\n",
        "\n",
        "Business workload (BI copilot / ETL assistant / app feature)\n",
        "→ Azure Databricks Model Serving endpoint\n",
        "→ Usage metadata capture (tokens, latency, endpoint, team)\n",
        "→ FinOps / chargeback table\n",
        "→ Platform decisions (routing, quotas, scale-to-zero, ownership)\n",
        "\n",
        "Operational controls such as endpoint config, tags, and access policy sit alongside the serving layer and shape the real economics of production AI."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "sample_events = [\n",
        "    {\"workload_class\": \"bi-copilot\", \"endpoint\": \"databricks-gpt-5-6\", \"cost_center\": \"finance-analytics\", \"latency_ms\": 820, \"total_tokens\": 1450},\n",
        "    {\"workload_class\": \"batch-enrichment\", \"endpoint\": \"databricks-gpt-5-6\", \"cost_center\": \"marketing-ops\", \"latency_ms\": 2400, \"total_tokens\": 9800},\n",
        "    {\"workload_class\": \"evaluation\", \"endpoint\": \"databricks-gpt-5-6\", \"cost_center\": \"ml-platform\", \"latency_ms\": 1300, \"total_tokens\": 4200},\n",
        "    {\"workload_class\": \"prod-api\", \"endpoint\": \"databricks-gpt-5-6\", \"cost_center\": \"digital-product\", \"latency_ms\": 690, \"total_tokens\": 1100}\n",
        "]\n",
        "\n",
        "df = spark.createDataFrame(sample_events)\n",
        "display(df.groupBy(\"workload_class\", \"cost_center\").agg(\n",
        "    F.count(\"*\").alias(\"calls\"),\n",
        "    F.avg(\"latency_ms\").alias(\"avg_latency_ms\"),\n",
        "    F.sum(\"total_tokens\").alias(\"sum_total_tokens\")\n",
        ").orderBy(\"workload_class\"))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Summary\n",
        "\n",
        "This notebook validated the blog's main claim: frontier model adoption on Azure Databricks is as much a platform economics decision as a model decision. By capturing usage metadata at inference time, persisting lightweight governance records, and inspecting endpoint ownership and controls, teams can make spend, accountability, and operational tradeoffs visible early.\n",
        "\n",
        "## Next Steps\n",
        "\n",
        "- Decide where inference belongs before teams benchmark models\n",
        "- Define workload classes before approving endpoints\n",
        "- Standardize usage logging with endpoint, latency, tokens, owner, and cost center\n",
        "- Build one chargeback view across interactive, batch, evaluation, and production API workloads\n",
        "- Review pilots older than 90 days as production until proven otherwise"
      ]
    }
  ]
}