{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.13.0"
    },
    "blog_metadata": {
      "topic": "Azure Cosmos DB Partition Key Strategy: Choosing the Right Approach Before You Scale",
      "slug": "azure-cosmos-db-partition-key-strategy-choosing-the-right-ap",
      "generated_by": "LinkedIn Post Generator + Azure OpenAI",
      "generated_at": "2026-07-09T00:13:57.082Z"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Azure Cosmos DB Partition Key Strategy: Choosing the Right Approach Before You Scale\n",
        "\n",
        "Partition keys in Azure Cosmos DB are not a late-stage tuning detail; they shape throughput distribution, storage distribution, query locality, transactional scope, and hotspot behavior from the start. This notebook turns the article into hands-on validation so you can test candidate partition keys with synthetic workloads before committing to a production container design."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "%pip install -q pandas matplotlib seaborn"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "from collections import Counter, defaultdict\n",
        "import random\n",
        "import math\n",
        "import statistics as stats\n",
        "\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "sns.set_theme(style='whitegrid')\n",
        "pd.set_option('display.max_rows', 20)\n",
        "pd.set_option('display.width', 120)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Decision flow for partition key evaluation\n",
        "\n",
        "This cell captures the article's decision loop in a notebook-friendly form. It gives you a compact checklist: generate representative workload data, evaluate candidate keys, measure skew and concentration, and iterate if risk is too high."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "decision_flow = {\n",
        "    'start': 'Sample workload events',\n",
        "    'steps': [\n",
        "        'Evaluate candidate partition keys',\n",
        "        'Measure cardinality and top-key concentration',\n",
        "        'Estimate skew and hot-partition risk',\n",
        "        'If acceptable, adopt key and validate with production metrics',\n",
        "        'If not acceptable, try synthetic or hierarchical key and repeat'\n",
        "    ]\n",
        "}\n",
        "\n",
        "for i, step in enumerate([decision_flow['start']] + decision_flow['steps'], start=1):\n",
        "    print(f'{i}. {step}')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Build representative sample event data\n",
        "\n",
        "The article emphasizes that average behavior rarely breaks systems; concentrated behavior does. This example creates a synthetic event stream with intentionally uneven tenant distribution so you can inspect how candidate partition keys behave under skew."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# Build sample event data to test candidate partition keys before production\n",
        "from collections import Counter\n",
        "import random\n",
        "\n",
        "random.seed(7)\n",
        "tenants = ['t1'] * 55 + ['t2'] * 25 + ['t3'] * 15 + ['t4'] * 5\n",
        "regions = ['us', 'eu', 'apac']\n",
        "events = []\n",
        "\n",
        "for i in range(1000):\n",
        "    tenant = random.choice(tenants)\n",
        "    region = random.choice(regions)\n",
        "    user_id = f'user-{random.randint(1, 250)}'\n",
        "    events.append({\n",
        "        'id': str(i),\n",
        "        'tenantId': tenant,\n",
        "        'region': region,\n",
        "        'userId': user_id,\n",
        "        'eventType': random.choice(['view', 'click', 'checkout']),\n",
        "    })\n",
        "\n",
        "print(events[:3])\n",
        "print('total_events =', len(events))\n",
        "\n",
        "sample_df = pd.DataFrame(events)\n",
        "display(sample_df.head())\n",
        "display(sample_df['tenantId'].value_counts().rename_axis('tenantId').reset_index(name='events'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Compare candidate keys for cardinality and skew\n",
        "\n",
        "This example tests three candidate keys and reports distinct key count, hottest key volume, top-share skew, and a simple risk label. The goal is not to model Cosmos DB internals exactly, but to quickly expose obviously risky choices before production."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# Compare candidate partition keys for cardinality, skew, and hot-partition risk\n",
        "from collections import Counter\n",
        "\n",
        "events = [{'tenantId': 't1', 'region': 'us', 'userId': 'u1'},\n",
        "          {'tenantId': 't1', 'region': 'us', 'userId': 'u2'},\n",
        "          {'tenantId': 't2', 'region': 'eu', 'userId': 'u3'},\n",
        "          {'tenantId': 't1', 'region': 'us', 'userId': 'u4'},\n",
        "          {'tenantId': 't3', 'region': 'apac', 'userId': 'u5'}] * 200\n",
        "\n",
        "candidates = {\n",
        "    '/tenantId': lambda e: e['tenantId'],\n",
        "    '/region': lambda e: e['region'],\n",
        "    '/tenantId#userId': lambda e: f\"{e['tenantId']}#{e['userId']}\",\n",
        "}\n",
        "\n",
        "rows = []\n",
        "for name, selector in candidates.items():\n",
        "    counts = Counter(selector(e) for e in events)\n",
        "    total = sum(counts.values())\n",
        "    hottest = counts.most_common(1)[0][1]\n",
        "    skew = round(hottest / total, 3)\n",
        "    risk = 'HIGH' if skew > 0.2 else 'MEDIUM' if skew > 0.1 else 'LOW'\n",
        "    print(f'{name:16} keys={len(counts):4} hottest={hottest:4} skew={skew:>5} risk={risk}')\n",
        "    rows.append({'candidate': name, 'keys': len(counts), 'hottest': hottest, 'skew': skew, 'risk': risk})\n",
        "\n",
        "comparison_df = pd.DataFrame(rows).sort_values(['skew', 'keys'], ascending=[True, False])\n",
        "display(comparison_df)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Recommend a candidate using simple guardrails\n",
        "\n",
        "This example turns the evaluation into a lightweight recommendation exercise. It ranks candidates by lowest top-share concentration and then by higher cardinality, which is a practical forcing function for architecture review discussions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# Recommend a partition key from sample events using simple guardrail thresholds\n",
        "from collections import Counter\n",
        "\n",
        "events = [{'tenantId': 't1', 'region': 'us', 'userId': f'u{i%50}'} for i in range(700)]\n",
        "events += [{'tenantId': 't2', 'region': 'eu', 'userId': f'u{i%200}'} for i in range(200)]\n",
        "events += [{'tenantId': 't3', 'region': 'apac', 'userId': f'u{i%100}'} for i in range(100)]\n",
        "\n",
        "def evaluate(name, fn):\n",
        "    counts = Counter(fn(e) for e in events)\n",
        "    total = sum(counts.values())\n",
        "    hottest = counts.most_common(1)[0][1]\n",
        "    cardinality = len(counts)\n",
        "    top_share = hottest / total\n",
        "    return {'name': name, 'cardinality': cardinality, 'top_share': round(top_share, 3)}\n",
        "\n",
        "results = [evaluate('/tenantId', lambda e: e['tenantId']),\n",
        "           evaluate('/region', lambda e: e['region']),\n",
        "           evaluate('/tenantId#userId', lambda e: f\"{e['tenantId']}#{e['userId']}\")]\n",
        "\n",
        "best = sorted(results, key=lambda r: (r['top_share'], -r['cardinality']))[0]\n",
        "print('candidates =', results)\n",
        "print('recommended =', best)\n",
        "\n",
        "display(pd.DataFrame(results).sort_values(['top_share', 'cardinality'], ascending=[True, False]))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Simulate RU concentration and hotspot risk\n",
        "\n",
        "The article warns that adding more overall RU/s does not automatically fix a hot logical partition. This simulation aggregates synthetic RU cost by candidate key so you can see whether load remains concentrated on a small set of values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# Simulate RU concentration by partition key and flag hot-partition candidates\n",
        "from collections import defaultdict\n",
        "import random\n",
        "\n",
        "random.seed(11)\n",
        "events = []\n",
        "for i in range(1500):\n",
        "    tenant = random.choice(['t1'] * 70 + ['t2'] * 20 + ['t3'] * 10)\n",
        "    ru_cost = random.choice([3, 5, 8, 13])\n",
        "    events.append({'tenantId': tenant, 'userId': f\"u{random.randint(1, 400)}\", 'ru': ru_cost})\n",
        "\n",
        "def score(events, key_fn):\n",
        "    ru_by_key = defaultdict(int)\n",
        "    for e in events:\n",
        "        ru_by_key[key_fn(e)] += e['ru']\n",
        "    total_ru = sum(ru_by_key.values())\n",
        "    hottest_key, hottest_ru = max(ru_by_key.items(), key=lambda kv: kv[1])\n",
        "    share = hottest_ru / total_ru\n",
        "    return hottest_key, hottest_ru, round(share, 3), ('HOT' if share > 0.2 else 'OK')\n",
        "\n",
        "rows = []\n",
        "for label, fn in {\n",
        "    '/tenantId': lambda e: e['tenantId'],\n",
        "    '/tenantId#userId': lambda e: f\"{e['tenantId']}#{e['userId']}\",\n",
        "}.items():\n",
        "    result = score(events, fn)\n",
        "    print(label, result)\n",
        "    rows.append({'candidate': label, 'hottest_key': result[0], 'hottest_ru': result[1], 'share': result[2], 'status': result[3]})\n",
        "\n",
        "ru_df = pd.DataFrame(rows)\n",
        "display(ru_df)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Visualize skew on the synthetic workload\n",
        "\n",
        "A chart often makes concentration risk easier to explain to reviewers than raw counts alone. This cell plots event counts by tenant from the earlier synthetic dataset to show how a seemingly simple key like `/tenantId` can inherit workload imbalance."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "random.seed(7)\n",
        "tenants = ['t1'] * 55 + ['t2'] * 25 + ['t3'] * 15 + ['t4'] * 5\n",
        "regions = ['us', 'eu', 'apac']\n",
        "events = []\n",
        "for i in range(1000):\n",
        "    tenant = random.choice(tenants)\n",
        "    region = random.choice(regions)\n",
        "    user_id = f'user-{random.randint(1, 250)}'\n",
        "    events.append({\n",
        "        'id': str(i),\n",
        "        'tenantId': tenant,\n",
        "        'region': region,\n",
        "        'userId': user_id,\n",
        "        'eventType': random.choice(['view', 'click', 'checkout']),\n",
        "    })\n",
        "\n",
        "plot_df = pd.DataFrame(events)\n",
        "counts = plot_df['tenantId'].value_counts().sort_values(ascending=False)\n",
        "\n",
        "plt.figure(figsize=(7, 4))\n",
        "ax = sns.barplot(x=counts.index, y=counts.values, palette='Blues_d')\n",
        "ax.set_title('Synthetic workload skew by tenant')\n",
        "ax.set_xlabel('tenantId')\n",
        "ax.set_ylabel('event count')\n",
        "for i, v in enumerate(counts.values):\n",
        "    ax.text(i, v + 5, str(v), ha='center', fontsize=9)\n",
        "plt.tight_layout()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Model the production feedback loop\n",
        "\n",
        "The article also highlights that production metrics should be treated as architecture signals, not just operations signals. This cell converts the sequence diagram into a simple ordered list showing how application traffic, Cosmos DB behavior, and monitoring data feed partition strategy reassessment."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "feedback_loop = [\n",
        "    ('Application', 'Cosmos DB Container', 'Writes/queries using chosen partition key'),\n",
        "    ('Cosmos DB Container', 'Application', 'RU charge + latency + 429 signals'),\n",
        "    ('Cosmos DB Container', 'Azure Monitor', 'Emit metrics and diagnostics'),\n",
        "    ('Azure Monitor', 'Application', 'RU, throttling, availability trends'),\n",
        "    ('Application', 'Application', 'Reassess partition strategy if hotspots emerge'),\n",
        "]\n",
        "\n",
        "for src, dst, msg in feedback_loop:\n",
        "    print(f'{src} -> {dst}: {msg}')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Required variables for live Azure metric collection\n",
        "\n",
        "If you adapt the next examples to query real Azure Monitor data, you will need values for:\n",
        "\n",
        "- `SUBSCRIPTION_ID`\n",
        "- `RESOURCE_GROUP`\n",
        "- `ACCOUNT_NAME`\n",
        "\n",
        "The notebook below uses Python placeholders and sample data so it remains runnable without Azure credentials."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Summarize partition-related metric spikes\n",
        "\n",
        "The original article included PowerShell for Azure Monitor. Because this notebook is Python-first, the logic is translated into Python using sample metric data so you can validate the review pattern without requiring a live Azure session."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "sample_metrics = [\n",
        "    {\n",
        "        'MetricName': 'NormalizedRUConsumption',\n",
        "        'Data': [{'Maximum': 61}, {'Maximum': 74}, {'Maximum': 88}, {'Maximum': 79}]\n",
        "    },\n",
        "    {\n",
        "        'MetricName': 'ServerSideLatency',\n",
        "        'Data': [{'Maximum': 12}, {'Maximum': 15}, {'Maximum': 18}, {'Maximum': 14}]\n",
        "    },\n",
        "    {\n",
        "        'MetricName': 'ThrottledRequests',\n",
        "        'Data': [{'Maximum': 0}, {'Maximum': 0}, {'Maximum': 3}, {'Maximum': 1}]\n",
        "    },\n",
        "]\n",
        "\n",
        "summary = []\n",
        "for metric in sample_metrics:\n",
        "    peak = max(point['Maximum'] for point in metric['Data'])\n",
        "    alert = 'Investigate' if metric['MetricName'] == 'ThrottledRequests' and peak > 0 else 'OK'\n",
        "    summary.append({'Metric': metric['MetricName'], 'Peak': peak, 'Alert': alert})\n",
        "\n",
        "summary_df = pd.DataFrame(summary)\n",
        "display(summary_df)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create a simple guardrail report\n",
        "\n",
        "This final validation example applies threshold-based guardrails to metric peaks. It is intentionally simple, but useful for standardizing architecture reviews around sustained RU pressure and throttling symptoms that may indicate partition-key problems."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "RuThreshold = 80\n",
        "ThrottleThreshold = 1\n",
        "\n",
        "sample = [\n",
        "    {'Metric': 'NormalizedRUConsumption', 'Peak': 92},\n",
        "    {'Metric': 'ThrottledRequests', 'Peak': 4},\n",
        "    {'Metric': 'ServerSideLatency', 'Peak': 18},\n",
        "]\n",
        "\n",
        "report = []\n",
        "for row in sample:\n",
        "    status = 'OK'\n",
        "    if row['Metric'] == 'NormalizedRUConsumption' and row['Peak'] >= RuThreshold:\n",
        "        status = 'Review partition strategy'\n",
        "    if row['Metric'] == 'ThrottledRequests' and row['Peak'] >= ThrottleThreshold:\n",
        "        status = 'Hot partition risk'\n",
        "    report.append({'Metric': row['Metric'], 'Peak': row['Peak'], 'Status': status})\n",
        "\n",
        "report_df = pd.DataFrame(report)\n",
        "display(report_df)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Summary\n",
        "\n",
        "This notebook validated the article's core claim: partition keys are a scaling contract, not a cosmetic schema choice. Using synthetic workloads, you tested candidate keys for cardinality, skew, hottest-key concentration, and RU hotspot risk, then translated production-monitoring ideas into repeatable Python guardrails.\n",
        "\n",
        "## Next Steps\n",
        "\n",
        "- Replace synthetic events with sampled production-like workload traces.\n",
        "- Add your real top read paths, write paths, backfills, retention jobs, and admin workflows to the evaluation.\n",
        "- Test hierarchical or synthetic keys when a single ownership dimension is too skewed.\n",
        "- Connect the metric review pattern to Azure Monitor or exported diagnostics.\n",
        "- Document a fallback migration plan before the first production launch."
      ]
    }
  ]
}