A practical, deep-dive map of Claude Code — what every folder, file, plugin, skill, hook, agent, and MCP actually does, how to compose them, how to swap the brain for OpenRouter or Ollama, and how it stacks up against Mastra and Pi.

Note: in this post, "Claude Code" refers to Anthropic's terminal coding harness. I'll use "harness" a lot — it's the cockpit that wraps the LLM and gives it tools, memory, and a runtime.

Why think in layers?

Picture Claude Code as a layered cake — or, if you prefer, an apartment building.

• The LLM is the foundation slab — raw intelligence with no plumbing.

• The harness is the building — walls, wiring, elevators, fire alarms.

• Each folder layer is a floor with its own residents (config, memory, plugins).

• Skills, subagents, MCPs, and hooks are the appliances and staff that actually get work done on each floor.

When something behaves weirdly — Claude forgets context, burns tokens, picks the wrong tool — it's almost always because you don't yet have a clear mental model of which floor the bug is on. The point of this post is to give you that floor plan.

Diagram 1 — The layered stack (top = most specific, bottom = rawest):

The full stack at a glance

From bottom (rawest) to top (most project-specific):

• Layer 0 — The LLM. Claude Sonnet, Opus, Haiku, or any model you route to.

• Layer 1 — The Harness. Claude Code itself — the loop that takes your prompt, decides on tool calls, edits files, runs bash.

• Layer 2 — ~/.claude/ (home). Your global settings, plugins, memory, and agents — applies to every project on this machine.

• Layer 3 — .claude/ (project). Repo-level config that travels with the codebase and is shared with your team.

• Layer 4 — Memory files. CLAUDE.md (committed) and CLAUDE.local.md (gitignored, personal).

• Layer 5 — Plugins. Installable bundles of skills, agents, hooks, commands, and MCP servers.

• Layer 6 — Skills. SKILL.md files that teach Claude how to do a specific task well.

• Layer 7 — Subagents. Specialized Claudes you dispatch with their own context window.

• Layer 8 — Hooks. Middleware that fires on lifecycle events (PreToolUse, PostToolUse, SessionStart, …).

• Layer 9 — MCPs. External tool servers (Slack, Linear, Notion, your DB, your CI, …).

Mental model: the harness loads layers in order, and later layers override earlier ones. Project beats home. Local beats committed. Skills beat raw prompts. That's the whole game.

Layer 0 — The LLM (the brain)

• This is the model weights themselves: Claude Opus 4.6, Sonnet 4.6, Haiku 4.5, etc.

• The LLM has no idea your project exists until the harness ships it the right context.

• It is swappable. The harness doesn't care which brain it's talking to as long as the API is Anthropic-compatible.

Alternatives:

• Hosted frontier: GPT-5 / o-series (OpenAI), Gemini 2.5 (Google), Grok (xAI), Mistral Large.

• Open-weights, local: Llama 4 / 3.x (Meta), Qwen 3 (Alibaba), DeepSeek-R1 / V3, Mixtral.

• Aggregators: OpenRouter (one key, 100+ models), Together AI, Fireworks, Groq.

Real-world analogy: the LLM is a brilliant freelance contractor. Smart, but useless without the keys to the building, the tools, and a brief.

Layer 1 — The Harness (Claude Code itself)

A harness is the runtime around the LLM:

• The agentic loop — read prompt → think → call tool → observe → repeat until done.

• The tool layer — Read, Write, Edit, Bash, Grep, Glob, WebFetch.

• The context manager — what's in the prompt, what got truncated, what's cached.

• The safety/permission layer — which folders Claude can touch, which commands need approval.

Alternatives:

• Terminal harnesses: Mastra Code (TypeScript-extensible, "never compacts"), Pi (pi.dev, minimalist), Aider (mature, token-efficient), Codex CLI (OpenAI), Gemini CLI (Google), Cline / Roo Code, OpenCode.

• IDE harnesses: Cursor, Continue.dev, Windsurf, Zed AI, JetBrains AI.

• Background / multi-agent: Devin, Factory, Sweep, OMC (oh-my-claudecode multi-CLI orchestration).

Analogy: if the LLM is the contractor, the harness is the work order, the toolbox, and the foreman combined. Same contractor, different harness, completely different output.

Diagram 2 — Harness anatomy:

Layer 2 — ~/.claude/ (your home base)

Lives in your home directory. Everything here applies to every project on this machine.

Typical contents:

• ~/.claude/settings.json — global preferences, default model, hook config.

• ~/.claude/CLAUDE.md — global memory ("I prefer pnpm over npm. I work in PT.").

• ~/.claude/agents/ — your personal subagents.

• ~/.claude/commands/ — slash commands available everywhere.

• ~/.claude/plugins/ — installed plugins.

• ~/.claude/hooks/ — your global hooks.

Alternatives (same idea, different tool):

• ~/.codex/ and ~/.codex/AGENTS.md (Codex CLI).

• ~/.gemini/ and GEMINI.md (Gemini CLI).

• ~/.aider.conf.yml + ~/.aider/ (Aider).

• ~/.cursor/rules/ + ~/.cursor/mcp.json (Cursor).

• ~/.continue/ (Continue.dev).

• ~/.config/mastra/ (Mastra Code).

Analogy: this is your personal toolbox in the truck. You bring it to every job site.

Layer 3 — Project .claude/ folder

A .claude/ directory inside your repo. Same shape as ~/.claude/, but scoped to this project and committed to git so the team gets the same setup.

• .claude/agents/ — project-specific subagents (e.g. db-migration-reviewer).

• .claude/commands/ — project slash commands (/deploy-staging).

• .claude/skills/ — project skills (e.g. "how we write Postgres migrations").

• .claude/hooks/ — project hooks (e.g. block commits without a Linear ticket).

• .claude/settings.json — project model overrides, allowed tools.

Alternatives (same idea, different tool):

• .cursor/rules/ and .cursor/mcp.json (Cursor).

• .codex/ (Codex CLI).

• .gemini/ (Gemini CLI).

• .aider/ + .aider.conf.yml + CONVENTIONS.md (Aider).

• .continue/ (Continue.dev).

• .mastra/ (Mastra Code, project-scoped threads + config).

• .vscode/settings.json for AI extensions.

Analogy: the shared toolbox in the workshop that everyone on the team uses for this job.

Layer 4 — The CLAUDE.md family (memory)

This is the layer most people get wrong. There are three flavors and they cascade:

• ~/.claude/CLAUDE.md — your global memory. Survives across all projects.

• <repo>/CLAUDE.md — project memory. Committed to git. Team-shared.

• <repo>/CLAUDE.local.md — your personal project notes. Gitignored. Don't commit.

Order of precedence at session start (most general → most specific):

~/.claude/CLAUDE.md → <repo>/CLAUDE.md → <repo>/CLAUDE.local.md

What actually goes in each:

• Global: your preferences, package manager defaults, communication style.

• Project committed: architecture overview, repo conventions, "we use Drizzle, not Prisma," critical gotchas.

• Project local: your local DB creds path, in-progress refactor notes, "don't touch X until Friday."

Alternatives (the "memory file" name in other harnesses):

• AGENTS.md — increasingly the cross-tool standard. Used by OpenAI Codex, Cursor, Aider, and others; many tools now read both AGENTS.md and their own file.

• GEMINI.md — Gemini CLI's equivalent.

• .cursorrules (legacy) → .cursor/rules/*.md (current) — Cursor's project-rules format.

• CONVENTIONS.md — Aider's preferred file (--read CONVENTIONS.md).

• System prompt files — Continue.dev and most TypeScript frameworks (Mastra) put this in code.

Pro tip: many teams now keep one AGENTS.md and symlink CLAUDE.md / GEMINI.md to it so every harness picks it up.

Analogy: think of CLAUDE.md as the onboarding doc you'd hand a new contractor. Global = "how I work." Project = "how the team works." Local = "what's weird about my setup right now."

Diagram 3 — The CLAUDE.md cascade:

Layer 5 — Plugins

A plugin is a packaged bundle that can include any combination of: skills, subagents, slash commands, hooks, and MCP servers. Install one plugin and you get a coordinated set of capabilities.

Examples you can install today:

• Superpowers — discipline plugin: TDD, systematic debugging, brainstorming, plan-writing, parallel agents. Very opinionated — it forces you into rigorous workflows.

• oh-my-claudecode (OMC) — multi-agent orchestration plugin. ~19 specialized agents, ~36 skills, runs Claude / Codex / Gemini in parallel tmux panes for cross-validation. Marketed as zero-config + 30–50% token savings.

• gstack — Garry Tan's (YC President) personal Claude Code stack, 23 opinionated tools modeling CEO / Designer / Eng Manager / Release Manager / Doc Engineer / QA roles. Workflow: Think → Plan → Build → Review → Test → Ship → Reflect.

• caveman — token-reduction plugin (more on this below).

Alternatives (plugin/extension ecosystems in other harnesses):

• Cursor — Cursor Rules library + Cursor extensions marketplace.

• Continue.dev — config.yaml blocks + community config presets.

• Aider — no plugin system by design; you compose via CONVENTIONS.md + git hooks.

• Codex CLI — slash-command bundles, AGENTS.md presets.

• Pi — first-class pi packages (third-party harness extensions).

• Mastra Code — extend programmatically with custom modes, tools, subagents in TypeScript.

Analogy: plugins are like buying a kitchen appliance bundle instead of one mixer at a time — they come pre-wired together.

Layer 6 — Skills (the secret sauce)

A skill is a folder with a SKILL.md file (and optional helper scripts/templates). It tells Claude how to do a specific task — the recipe, not the ingredients.

• Skills load on demand when their description matches your request — that's "progressive disclosure" and it's why skills don't blow up your context window.

• Skills can be standalone in .claude/skills/ or bundled inside plugins.

• A great skill is a checklist + a few examples + a link to deeper docs the LLM can read if needed.

Alternatives (the "on-demand instructions" pattern):

• Cursor — Project Rules / User Rules with glob-based scoping.

• Pi — Skills are first-class, designed not to bust the prompt cache.

• Mastra Code — Skills + custom commands.

• gstack — slash commands as opinionated mini-skills.

• Aider — no equivalent; you'd inline guidance in CONVENTIONS.md.

• Continue.dev — prompt template files + .prompt blocks.

Analogy: skills are standard operating procedures posted on the wall. The contractor (LLM) reads the relevant SOP only when they're about to do that task — they don't memorize all of them up front.

Layer 7 — Subagents

A subagent is a separate Claude instance you dispatch with a focused task and a fresh context window. It returns a summary; the bulk of its working memory never pollutes yours.

Why this matters:

• Context isolation — a 50k-token research dive doesn't bloat your main session.

• Specialization — code-reviewer, Explore, Plan, custom domain reviewers.

• Parallelism — fire several at once for independent tasks.

Alternatives (multi-agent / delegation patterns):

• Mastra Code — custom subagents in TypeScript via the Agent primitive.

• Mastra (framework) — full agent + workflow graph (DAGs, branches, parallelism).

• OMC (oh-my-claudecode) — runs Claude + Codex + Gemini as parallel CLI panes for cross-validation.

• Cline / Roo Code — built-in "Plan/Act" modes acting as lightweight subagents.

• CrewAI / AutoGen / LangGraph — framework-land alternatives if you're building outside a CLI.

• Pi — none by default (philosophical); add via packages.

• Aider — single-agent; uses architect/editor roles on one model instead.

Analogy: delegating to a specialist consultant. They go off, do the deep work, and email you a one-pager.

Diagram 4 — Subagent fan-out (main session delegates, gets summaries back):

Layer 8 — Hooks (the middleware)

Hooks are scripts that run automatically on lifecycle events. They're the interceptors of Claude Code.

Common hook points:

• SessionStart — runs when you launch Claude Code (e.g. inject today's date, current branch, recent commits).

• PreToolUse — fires before any tool call. Can block, modify args, or warn.

• PostToolUse — fires after. Great for auto-formatting, logging, or running tests after Edit.

• UserPromptSubmit — fires when you send a message. Inject extra context here.

• Stop / SubagentStop — cleanup, summarization, notifications.

What they're great for:

• Auto-running prettier / ruff after every Edit.

• Blocking writes outside the project root.

• Posting a Slack message when a long task finishes.

• Injecting "what changed since my last commit" into every session.

Alternatives (lifecycle / middleware in other tools):

• Mastra Code — has its own hooks layer (pre/post tool, session lifecycle).

• Aider — built-in lint/test commands + git hooks at the OS layer.

• Cursor — auto-format-on-save and limited "background tasks."

• Pi — extend via skills + scripts (no formal hooks API).

• OS / git hooks — pre-commit, pre-push, Husky — work with any harness.

• Continue.dev — slash commands + indexer hooks.

Analogy: hooks are the building's automation system — motion-activated lights, fire alarms, key-card locks. Claude doesn't think about them; they just fire on the right event.

Layer 9 — MCPs (external tools)

Model Context Protocol servers expose external systems to Claude as callable tools — Slack, Linear, Notion, GitHub, your database, your internal API, etc.

• An MCP server defines tools + their JSON schemas.

• The harness loads them, hands the schemas to the LLM, and proxies calls.

• Plugins frequently bundle MCPs (e.g. a "PM plugin" might bundle Linear + Notion + Figma MCPs).

Alternatives (how each ecosystem exposes external tools):

• MCP (cross-tool standard) — supported by Claude Code, Cursor, Mastra Code, Continue, Codex, Gemini CLI, Cline, and most modern harnesses. The closest thing to a "USB-C for AI tools."

• OpenAI function calling / Tools API — the older, ecosystem-specific equivalent.

• LangChain / LlamaIndex tools — framework-side abstractions.

• Mastra tools — TypeScript-native tool definitions in the Mastra framework.

• Aider — minimal; relies on shell + git instead of external connectors.

• Custom REST/GraphQL — wrap anything as an MCP server with one of the SDKs.

Analogy: MCPs are the utility hookups — water, gas, electricity, internet. The building (harness) is useless without them once you want to do real work.

How everything stacks together

The thing that confuses people: all these layers are active at once. Here's the mental order on every turn:

1. The LLM receives a prompt.

2. The harness has prepended: system prompt + global CLAUDE.md + project CLAUDE.md + local CLAUDE.md + active skills' descriptions + tool schemas (incl. MCPs).

3. Hooks may have already run (SessionStart, UserPromptSubmit) and injected extra context.

4. The LLM picks a tool. PreToolUse hooks fire — they can block or modify.

5. Tool runs. PostToolUse hooks fire.

6. If the task is gnarly, the LLM dispatches a subagent, which runs the same loop in isolation.

7. Repeat until done. Stop hooks fire.

Plugins like Superpowers + your own agents play nicely together because they live in different layers: Superpowers ships skills + commands; your custom agents live in .claude/agents/; your hooks live in .claude/hooks/. They compose because the harness merges them at load time.

Diagram 5 — One full turn, including hooks firing:

The framework ecosystem (the "configs on top of configs")

This is where it gets interesting. People are building opinionated stacks on top of Claude Code, the way oh-my-zsh sits on top of zsh.

Superpowers

• A discipline-first plugin. Strong opinions on TDD, systematic debugging, plan-before-code, evidence-before-claims.

• Forces you to use the skill even when you think you don't need it.

• Best for: solo devs and small teams who want guardrails against "vibe coding."

oh-my-claudecode (OMC)

• Multi-agent orchestration. Ships ~19 agents and ~36 skills.

• Runs Claude + Codex + Gemini CLIs in parallel tmux panes — great for cross-checking architecture, security review, deep code analysis.

• Marketed at 30–50% token savings via specialization.

gstack (Garry Tan / YC)

• 23 tools modeling startup roles. The skills /office-hours, /review, /ship are the headline features.

• Workflow is the product: Think → Plan → Build → Review → Test → Ship → Reflect.

• Tan claims ~10K logical LOC/week throughput while running YC full-time. Take the number with a grain of salt; the workflow itself is the real artifact.

caveman

• Token-reduction skill. Forces Claude to drop articles, filler words, hedging — output reads like cave drawings.

• Reported ~65% drop in average response tokens (294 vs 1,214 across 11 dev tasks); spread of 22–87% depending on prompt.

• Bonus: a March 2026 paper ("Brevity Constraints Reverse Performance Hierarchies in Language Models") found brevity-constrained models gained up to 26pp in accuracy on some benchmarks. Less can literally be more.

All four can be combined. Superpowers for discipline + gstack for workflow + caveman for compression + your own .claude/ for project-specific glue.

Swapping the brain: OpenRouter and Ollama

Claude Code ships pointed at Anthropic's API, but the harness only cares about an Anthropic-compatible endpoint. Two main routes:

Option A — Claude Code Router (CCR)

• A community proxy that translates Claude Code's API calls to OpenRouter, OpenAI, Gemini, DeepSeek, Ollama, etc.

• You set ANTHROPIC_BASE_URL (and an API key) to point at the router.

• Per-task routing: cheap model for trivial edits, frontier model for hard reasoning.

Option B — LiteLLM proxy

• Same idea, more general-purpose. Spins up an OpenAI-/Anthropic-compatible endpoint that fans out to ~100+ providers.

Option C — Ollama (fully local)

• Run a model locally — Llama 3.x, Qwen, DeepSeek-R1 distills, etc.

• Stick LiteLLM (or CCR) in front of Ollama to expose an Anthropic-compatible API.

• Trade-off: zero API cost, full privacy; but tool-use quality varies wildly by model. Frontier-class agentic behavior is still a stretch on consumer hardware.

Quick recipe (conceptual):

• OLLAMA_HOST=... runs the model.

• LiteLLM exposes http://localhost:4000 as Anthropic-compatible.

• export ANTHROPIC_BASE_URL=http://localhost:4000 and export ANTHROPIC_AUTH_TOKEN=anything.

• Launch Claude Code. Same harness, different brain.

Analogy: the harness is the car. Claude is the default engine. OpenRouter and Ollama let you swap the engine without rebuilding the chassis. The steering wheel (your skills, hooks, CLAUDE.md) doesn't care which engine you bolted in.

⚠️ Caveat: smaller open models often struggle with multi-tool workflows, file editing precision, and long contexts. For local dev you're often best with a hybrid — Ollama for cheap autocomplete-like loops, Claude/Anthropic for the hard stuff.

Diagram 6 — Same harness, swappable brain:

Comparing harnesses: Claude Code vs Mastra Code vs Pi

You asked which harness uses fewer tokens. Short answer: it's not the harness, it's the discipline layered on top. But the three harnesses have real personality differences.

⚠️ Quick disambiguation. Mastra is the TypeScript framework for building AI apps and agents (think SDK + primitives: Harness, Agent, Memory). Mastra Code is a separate terminal coding agent built on top of Mastra's primitives — it's the direct apples-to-apples comparison to Claude Code. Below I'm comparing Mastra Code, the harness, not the framework.

Claude Code

• Heaviest by default — rich tool layer, MCPs, plugins, subagents, hooks all in one box.

• Best ergonomics for humans; biggest ecosystem (Superpowers, OMC, gstack, caveman).

• Token cost can creep up fast unless you use skills + caveman + subagents to isolate context.

• Memory model: prompt-based (CLAUDE.md cascade) + auto-compaction when the context window fills.

Mastra Code (code.mastra.ai)

• Terminal-based coding agent built on Mastra's Harness / Agent / Memory primitives.

• Headline feature: "never compacts." Uses observational memory — it watches, reflects, and compresses context as it goes, instead of doing a hard truncation when the window fills. The pitch: long-running sessions stay precise.

• Three explicit modes — Build / Plan / Fast — so you pick a different cost/quality profile per task.

• Supports MCP servers, hooks, custom commands, skills, project-scoped threads.

• Programmatically extensible: custom modes, tools, subagents, storage all in TypeScript.

• Connects to "thousands" of models out of the box (you bring the API key).

• Requires Node 22.13+.

• Strength: longest-running sessions stay coherent without manual /compact. Weakness: smaller ecosystem of community plugins than Claude Code today.

Pi (pi.dev, by Mario Zechner)

• Minimalist terminal coding harness. Explicit non-goal: features like sub-agents and plan mode are not built in — you add them only if you need them.

• Skills are first-class, loaded on-demand without breaking the prompt cache.

• Self-modifying philosophy: Pi can rewrite Pi to fit your workflow.

• Often the lowest token-per-task in head-to-head comparisons because there's less ambient harness overhead — but you pay for it in setup time.

Rough trade-off table

• Most plug-and-play → Claude Code.

• Best at long, never-compact sessions → Mastra Code (observational memory).

• Lowest baseline token spend → Pi (especially with caveman bolted on).

• Best for multi-agent orchestration → Claude Code with OMC, or Mastra Code with custom subagents.

• Easiest to extend in TypeScript → Mastra Code.

The honest take: no harness is intrinsically cheaper. What matters is (a) how much system prompt you stuff, (b) how aggressively you use subagents to isolate context, (c) whether your output style is verbose by default, and (d) whether your harness compacts (Claude Code) or compresses-as-it-goes (Mastra Code) — the latter avoids the cliff where Claude Code "forgets" mid-session.

Token economics (and where caveman fits)

Where tokens actually go in a Claude Code session:

• System prompt + harness instructions — fixed overhead per turn.

• CLAUDE.md cascade — grows with every doc you add.

• Active skills' descriptions — minor; only triggered skills load full content.

• Tool schemas (esp. MCPs) — often the biggest invisible cost. Loading 200 MCP tool schemas can dwarf your prompt.

• Conversation history — grows linearly. Caching helps.

• Output tokens — what caveman attacks.

Practical levers:

• Prune MCPs aggressively. If you don't use a connector this session, don't load it.

• Move long instructions into skills, not CLAUDE.md, so they only load on demand.

• Use subagents for research — keep their summary, not their working memory.

• Run caveman when your task is well-defined and you don't need verbose explanations.

• Cache the system prompt. Anthropic's prompt caching cuts repeat-prompt cost dramatically.

Putting it all together

You don't pick one layer — you compose them. A reasonable "power user" setup:

• Brain: Claude (default) for hard tasks, Ollama via LiteLLM for cheap loops.

• Harness: Claude Code.

• Home: ~/.claude/CLAUDE.md with your prefs, plus Superpowers + caveman installed globally.

• Project: .claude/ with team CLAUDE.md, project skills, a code-reviewer subagent, a pre-commit hook.

• Personal: CLAUDE.local.md for your in-progress quirks.

• Workflow: gstack-style Think → Plan → Build → Review → Test → Ship.

Analogy to close on: Claude Code isn't a tool — it's a building you can renovate. The LLM is rented power. The harness is the structure. Your .claude/ folders are the floors you actually live on. Plugins are pre-fab kitchens. Skills are recipes pinned to the fridge. Subagents are interns. Hooks are the smart-home automation. MCPs are the utilities.

You don't have to build everything yourself — but you do have to know which floor the light switch is on.

Sources & further reading

• caveman GitHub (JuliusBrussee)

• Caveman Claude Code: How to Save Tokens

• Claude Code Caveman Mode: Save >75% on Usage

• oh-my-claudecode (yeachan-heo)

• oh-my-claudecode website

• gstack (Garry Tan)

• GStack Tutorial — SitePoint

• What Is GStack? — MindStudio

• Pi Coding Agent (pi.dev)

• pi-mono on GitHub

• What I learned building an opinionated, minimal coding agent

• Mastra Code (code.mastra.ai)

• Announcing Mastra Code: A coding agent that never compacts

• Mastra Code Configuration docs

• Mastra (the framework) on GitHub

When I switched to Claude Code, it felt different. Not "oh cool, another copilot" different — more like going from a GPS that suggests routes to a co-driver who actually drives. 🚗

Here's everything I wish someone told me on Day 1.

First — How is Claude Code Different from Cursor?

Cursor is great. I've used it. But here's the honest comparison:

Cursor lives inside VS Code. It's an IDE-first experience — you get AI autocomplete, inline chat, and a composer tab. Think of it like having a smart assistant sitting inside your office.

Claude Code lives in your terminal. It's an agentic coding tool — it doesn't just suggest code, it executes things. It reads your codebase, runs commands, makes git commits, creates PRs, and manages files. Think of it like hiring a junior dev who actually does the work, not just tells you what to do. 🧑‍💻

Here's what really sets Claude Code apart:

→ Plan mode + Skills — Cursor has plan mode too, but Claude Code takes it further. You can Shift+Tab to cycle between Normal mode (full agent), Plan mode (read-only research — Claude looks but doesn't touch), and Auto-accept mode (full speed, no permission prompts). It's like having gears in a car — you pick the level of control you want.

→ Hooks — This is something Cursor doesn't have. Claude Code lets you run custom scripts triggered by events (before/after tool use, on conversation start, etc.). Think of it like GitHub Actions but inside your coding agent. Pre-commit linting, auto-formatting, custom checks — all automatic.

→ Skills over Commands — Claude Code used to have "commands" (.claude/commands/*.md) and "skills" (.claude/skills/*/SKILL.md) as separate things. They've now been unified — skills ARE the slash command system. Important: commands are deprecated — use skills going forward. Skills support additional features like supporting files, and Claude can even auto-trigger them based on context rather than you typing a slash command.

→ Subagents — Claude Code can spawn specialized agents (Explore, Plan, Verify) to handle different parts of a task in parallel. Cursor doesn't have this concept. It's like having a team of specialists instead of one generalist.

The Commands You Need to Know

I'm breaking these into levels because there's a lot, and you don't need everything on Day 1.

🟢 Basic (Start Here)

• /init — First thing you run in any project. It generates a CLAUDE.md file — basically a cheat sheet that tells Claude about your codebase, conventions, and preferences. Think of it like onboarding a new team member.

• /help — Shows all available commands and skills. Your compass when you're lost.

• /compact — Compresses your conversation context to save tokens (and money). Long conversation getting expensive? Hit compact. Like clearing your browser cache but for AI context.

• /clear — Nuclear option. Wipes all conversation history. File edits remain, but the slate is clean.

• /cost — See how much your current session is costing. Always good to check.

🟡 Intermediate (Once You're Comfortable)

• /batch — Orchestrates large-scale changes across your codebase in parallel. Need to update 50 files? Don't do them one by one. Batch is like having 10 devs working on different files simultaneously.

• /fork — Copies your current conversation into a branch without touching the original. Like git branch, but for your AI conversation.

• /rewind — Rolls back your conversation (including code changes) to an earlier point. Your "undo" button when Claude goes in the wrong direction.

• /review — Runs a code review on your changes before you commit. Like having a senior dev glance at your PR before you push.

• /mcp — Exposes MCP (Model Context Protocol) operations as slash commands. This is how you connect Claude to external tools — GitHub, Slack, databases, anything.

🔴 Advanced (Power User Territory)

• Permission Modes — Five levels, cycle with Shift+Tab:

  • Default — Prompts for every potentially dangerous operation

  • Auto-accept (acceptEdits) — No permission prompts for file edits. Full speed.

  • Plan mode — Read-only. Claude researches and plans but can't modify anything.

  • Don't Ask — Auto-denies everything not explicitly pre-approved

  • Bypass — Skips ALL permission checks. Only use in isolated/sandboxed environments.

• Subagents — Specialized agents you can spawn:

  • Explore — Fast codebase exploration (finding files, searching code)

  • Plan — Software architect for designing implementation strategies

  • Verify — Validation agent for checking work

• Skills (replacing commands) — Create .claude/skills/your-skill/SKILL.md files to extend Claude's capabilities. Skills can be auto-triggered or manually invoked via /your-skill. You can build anything — code reviewers, deployment pipelines, content generators, testing frameworks.

• /simplify — Spawns three parallel review agents that check your recent changes for code reuse, quality, and efficiency issues, then fixes them. Like running three senior reviewers at once. 🔥

• CLAUDE.md — Not a command, but the most powerful concept. This markdown file at your project root is Claude's memory. Put your coding conventions, architecture decisions, tech stack preferences — everything. It's like writing an employee handbook for your AI teammate.

GitHub Repos You Should Star Right Now ⭐

These are the repos that'll 10x your Claude Code game:

→ Awesome Claude Code — The mothership. Curated list of tools, skills, plugins, integrations, and resources. If you only star one repo, make it this one.

→ GStack by Garry Tan — 20K+ stars. Turns Claude Code into a virtual engineering team — CEO, designer, eng manager, QA lead, security officer, release engineer. 20 specialists, 8 power tools, all as slash commands. MIT license. It hit 10K stars in 48 hours. 🔥

→ Get Shit Done (GSD) — A spec-driven development workflow that prevents context rot. It externalizes state into files, splits work into small plans, executes each in fresh context, and verifies against explicit goals. Now a standalone CLI in v2.

→ Agency Agents — 112+ specialized AI agent personas you can inject into Claude Code (and other tools). Think of it like hiring an entire agency — frontend wizards, QA leads, security reviewers, community managers — each with their own personality, processes, and deliverables. Built for Claude Code natively, works with the .md + YAML frontmatter format out of the box.

→ Superset.sh — Want to run 10+ Claude Code agents in parallel? Superset orchestrates multiple Claude Code instances across isolated git worktrees — no merge conflicts, no stepping on each other's changes. It's like going from one developer to a whole engineering floor. Free tier available, Pro at $20/seat/month. (GitHub)

→ Claude Code Official Repo — The source. Star it, watch it, read the issues. This is where the roadmap lives.

Twitter/X Accounts to Follow 🐦

→ @bcherny (Boris Cherny) — The creator of Claude Code himself. He regularly shares tips, how he uses Claude Code, and behind-the-scenes insights from the team. His thread on "how I use Claude Code" is required reading. He started Claude Code as a side project in September 2024 — and today it writes 4% of all GitHub commits.

→ @AnthropicAI — Official Anthropic account for product announcements and updates.

→ @alexalbert__ — Alex Albert, head of Claude Relations at Anthropic. Great for understanding new features and model updates.

→ @garaborosz — Gábor, who curates a lot of Claude Code content and tutorials.

My Honest Take

Claude Code isn't trying to be Cursor. Cursor is an AI-enhanced IDE. Claude Code is an AI agent that happens to code.

If you want autocomplete and inline suggestions while you type → Cursor. If you want an agent that reads your repo, plans architecture, writes code, runs tests, makes commits, and creates PRs → Claude Code.

Some people use both. And that's fine too.

The biggest mistake I see people make: they install Claude Code and use it like ChatGPT — just asking questions. That's like buying a Tesla and only using it in park. 🐴

Run /init. Set up your CLAUDE.md. Build custom skills. Use plan mode. Let it drive.

The #1 Rule of Thumb 🧠

When in doubt — just ask Claude.

Seriously. Confused about a command? Ask Claude. Not sure which mode to use? Ask Claude. Don't know how to set up a skill? Ask Claude.

Claude Code is surprisingly good at explaining itself. It's like having a teammate who never gets annoyed when you ask "wait, how does this work again?" for the fifth time. Most people forget that the tool they're using IS the help desk. You don't need to Google "how to use Claude Code" — you can literally ask Claude Code.

If you don't know, if you're confused, if you're stuck — your first move should always be: ask Claude.

What's Next? Measure What Matters 📊

Once you're up and running with Claude Code, the next step is setting up hooks to measure your productivity. Think of it like a fitness tracker — you've started working out, now track the gains.

GStack makes this easy with two powerful commands:

→ /insights — Gives you a breakdown of what Claude did, how much code was generated, and where time was spent. It's your personal sprint dashboard.

→ /retro — Runs a retrospective on your session — what went well, what didn't, and what to improve. Like doing a standup with your AI teammate after every coding session.

But before you retrospect — you need to actually see what Claude is doing under the hood. Two flags that'll change how you debug:

→ --verbose — Shows you every tool call, every file read, every decision Claude makes in real time. Think of it like turning on "show my work" mode in a math exam. You see the reasoning, not just the answer. Great for understanding why Claude did what it did.

→ --debug — Goes even deeper. Logs API calls, token counts, timing info, and internal state. This is your X-ray machine. When something goes wrong and you don't know why, --debug is the first flag you reach for.

The workflow: Run your session with --verbose, then use /insights to see what happened, and /retro to reflect on it. It's like recording a game, watching the replay, and then doing a post-match analysis. 🎮

One more hidden gem:

→ /btw — This lets you inject a side note or additional context mid-conversation without interrupting Claude's current task. Think of it like passing a sticky note to someone while they're working — "btw, don't forget we use ESLint" or "btw, the API key is in .env.local." Claude absorbs the context and keeps going. Super useful when you remember something mid-flow.

You can't improve what you don't measure. Set up hooks, run --verbose to watch Claude work, /insights after every session, and /retro at the end of the week. That's how you go from "I use Claude Code" to "Claude Code makes me 10x." 🚀

What's your experience been with Claude Code? Drop your favorite command or skill in the comments — I'm always looking for new tricks. 👇

#ClaudeCode #AIAgents #DeveloperTools #CodingWithAI #Anthropic #BuildInPublic

credits : to the original creator

I did that too. Then things started breaking in weird ways — production code without tests, force-pushes to main at 2am, context getting wiped mid-session and Claude forgetting critical rules.

So I stopped and asked: what if the AI agent had guardrails that enforced discipline automatically — and measured themselves?

Over the past few weeks, I've built a hook system for Claude Code inside our fintech monorepo. Not a few scripts stitched together — a full lifecycle of pre-checks, post-validations, audit trails, context recovery, and a metrics layer that snapshots progress daily.

Here's what it looks like, what each hook does, and what the numbers are telling me.

What Are Claude Code Hooks?

Think of hooks like airport security checkpoints for your AI agent.

Before Claude runs a bash command → a pre-tool hook inspects it. After Claude edits a file → a post-tool hook validates it. When Claude's context gets compacted (memory reset) → a compact hook saves and restores critical rules.

Every hook runs automatically. Zero manual intervention. Claude doesn't even know it's being watched — it just gets blocked or warned when it tries something it shouldn't.

The Hook Categories (and Why Each One Exists)

🛡️ 1. Safety Hooks — The Seatbelts

Problem: AI agents can run any bash command. Including destructive ones.

Hook: pre-tool-bash — intercepts every bash command before execution and blocks patterns like:

→ rm -rf /           ← blocked
→ DROP DATABASE       ← blocked
→ git push --force main  ← blocked
→ redis-cli FLUSHALL     ← blocked

Real-world analogy: It's like a car that physically won't start if the seatbelt isn't on. You don't rely on the driver remembering — the system enforces it.

Impact: Zero destructive commands have gotten through. Ever.

🧪 2. TDD Enforcement — The Bouncer

Problem: It's tempting to write production code first and "add tests later." With AI agents, this temptation is 10x worse — Claude can generate code so fast that tests become an afterthought.

Hook: pre-tool-tdd — blocks any write to a .ts production file if the corresponding .spec.ts test file doesn't exist yet.

Claude tries to write: apps/smf-core/src/services/ledger.service.ts

Hook checks: does ledger.service.spec.ts exist?
  → No? BLOCKED. Exit code 2. Write rejected.
  → Yes? Proceed.

Smart enough to exempt files that don't need tests — type definitions, enums, constants, DTOs, barrel exports.

Real-world analogy: A bouncer at a club checking your ID. No test file = no entry. Doesn't matter how good the code looks.

Current metric: 100% TDD compliance. Every production file has its test. Not because I'm disciplined — because the hook won't let me skip it.

🔍 3. Auto-Typecheck — The Spell Checker

Problem: TypeScript errors compound fast in a monorepo. One bad type change ripples across 6 packages. By the time you notice, you've built 3 more features on a broken foundation.

Hook: post-tool-typecheck — runs a scoped typecheck on the affected workspace immediately after every .ts file edit.

Claude edits: apps/auth-service/src/auth.controller.ts

Hook triggers: pnpm turbo run typecheck --filter=auth-service
  → Pass? Silent. Continue working.
  → Fail? Warning with exact error. Fix before moving on.

Real-world analogy: Like spell-check highlighting errors as you type — not waiting until you hit "submit."

Why it matters: Catches type regressions within seconds of introduction, not hours later during a build.

🎯 4. Agent Model Governance — The Right Specialist for the Right Job

Problem: Our monorepo uses multiple Claude agents — Haiku for lightweight services (card-processor, auth), Sonnet for complex business logic (smf-core, integration tests), Opus for final PM review. Spawning the wrong model wastes tokens and time.

Hook: pre-tool-agent — validates that each spawned agent uses the expected model tier.

Spawning: card-processor agent
Expected model: haiku
Actual model: sonnet
  → WARNING: Model mismatch detected. Consider using haiku.

Real-world analogy: Like a hospital assigning the right specialist to the right procedure. You don't send the chief surgeon to check a blood pressure reading.

Current metric: 57.1% model match rate. Yes, that's low — and the hook is exactly how I know it's low. Without measurement, I'd have assumed it was fine. Now I'm actively improving it.

🧠 5. Context Recovery — Saving Your Game Before the Boss Fight

Problem: Claude Code compacts context when conversations get long. When it compacts, it can forget critical rules — like "always write tests first" or "never modify shared-types directly."

Hooks: pre-compact + post-compact-inject working as a pair.

BEFORE compaction:
  → pre-compact takes a snapshot:
    - Current status files
    - Git state
    - 5 critical rules

AFTER compaction:
  → post-compact-inject restores:
    - All 5 critical rules as additionalContext
    - The pre-compact snapshot for state recovery

The 5 critical rules that survive every compaction:

1. TDD: failing test first, then implementation

2. Typecheck after every edit

3. Never modify shared-types directly

4. Modern Treasury metadata must include smf_type

5. Phase 3 review is mandatory before commit

Real-world analogy: Like saving your game before a boss fight. If you die (context reset), you reload from the save point — not from the beginning.

Current metric: 6 compaction events survived with zero rule amnesia.

📝 6. Audit Trail — The Black Box Recorder

Problem: When something goes wrong in a long AI session, you need to trace what happened. "What commands did Claude run? What files did it touch? Which skills did it use?"

Hooks: Three loggers running in parallel:

command-logger  → logs every bash command (rolling 200 entries)
skill-logger    → logs every skill invocation (rolling 500 entries)
subagent-logger → logs every agent spawn (model, type, description)
write-audit     → logs every file write with timestamp

Plus a shared error trap (_err-trap) that captures hook failures with timestamp, exit code, and exact line number.

Real-world analogy: A flight data recorder (black box). You hope you never need it. But when you do, it tells you exactly what happened.

Current metric: 227 file operations logged. 200 commands tracked. 10 skill/agent invocations recorded. Every single one traceable.

🗼 7. Orchestration Hooks — Air Traffic Control

Problem: In a multi-agent monorepo, agents can block each other. One agent hits an unresolvable issue → the whole pipeline needs to halt. Another agent finishes → the next phase should begin.

Hook: subagent-stop — runs when any sub-agent completes:

Agent finishes → hook checks status files:
  → PROJECT_BLOCKED.md found? HALT EVERYTHING. Surface to human.
  → BLOCKED.md found? Log it. Attempt retry.
  → FIXED.md found? All clear. Advance to next phase.

Real-world analogy: Air traffic control. Every plane (agent) reports its status on landing. ATC (orchestrator) decides what takes off next.

The Metrics Layer — Hooks That Measure Themselves

Here's where it gets interesting.

The hooks don't just enforce rules — they generate structured telemetry. Every blocked command, every TDD violation, every model mismatch, every compaction event gets logged as a JSONL entry with timestamps.

Then a /metrics skill reads all that telemetry, computes 6 core metrics, compares against the previous day's snapshot, and shows trends:

┌──────────────────────┬─────────
│ Metric               │ Today   │ Trend           │
├──────────────────────┼─────────
│ Hook Reliability     │ 98.6%   │   →            │
│ TDD Compliance       │ 100%    │   →            │
│ Agent Model Match    │ 57.1%   │   ↑             │
│ Typecheck Pass Rate  │ pending │   —             │
│ Review First-Pass    │ pending │   —             │
│ Compaction Events    │ 6/week  │   →            │
└──────────────────────┴─────────

Daily snapshots get saved to .claude/metrics/snapshots/ — one JSON file per day. Over time, this builds a progress timeline showing exactly where discipline is improving and where it's slipping.

What the Numbers Are Telling Me

• 98.6% hook reliability across 427+ automated runs — 4 errors total, all from the agent model hook during early calibration. The system is stable.

• 100% TDD compliance — not a single production file exists without its test. The bouncer works.

• 57.1% agent model match — this is the weakest spot, and I only know because the hook is measuring it. Active improvement area.

• 6 compaction survivals — context got wiped 6 times, and all 5 critical rules were restored every time. Zero rule amnesia.

Supporting Skills in the Ecosystem

The metrics don't just sit in JSON files. They feed into a broader analysis ecosystem:

• /health-check — runs at session start, reports hook errors from the last 7 days

• /reflect — analyzes command logs and skill usage to suggest new automation rules

• /retro — weekly retrospective with week-over-week trend comparison

• analyze-patterns.sh — detects repeated commands (3+ times) and suggests turning them into skills

It's a feedback loop: hooks enforce → metrics measure → skills analyze → rules improve → hooks enforce better.

The Flywheel Effect

Here's the mental model I keep coming back to:

     ┌──────────────┐
     │   Hooks       │ ← enforce rules automatically
     │   (guardrails)│
     └──────┬───────┘
            │ generate
            ▼
     ┌──────────────┐
     │   Metrics     │ ← structured telemetry
     │   (JSONL)     │
     └──────┬───────┘
            │ feed into
            ▼
     ┌──────────────┐
     │   Skills      │ ← /metrics, /reflect, /retro
     │   (analysis)  │
     └──────┬───────┘
            │ suggest
            ▼
     ┌──────────────┐
     │   Rules       │ ← CLAUDE.md, new hooks
     │   (improved)  │
     └──────┬───────┘
            │ enforced by
            ▼
     ┌──────────────┐
     │   Hooks       │ ← cycle repeats, tighter each time
     └──────────────┘

Every cycle, the system gets tighter. The agent gets more disciplined. The code gets more reliable. And I have the numbers to prove it's actually happening.

What I'd Tell Someone Starting This Today

• Start with safety hooks. Block destructive commands before anything else. It takes 10 minutes and saves you from catastrophic mistakes.

• Add TDD enforcement early. Once you have 500 files without tests, it's too late. The bouncer is easiest to install when the club is empty.

• Measure everything, even the embarrassing numbers. My 57% model match rate isn't pretty. But I can only improve what I can see.

• Build the compact recovery hooks. Context loss is the silent killer of long AI sessions. Your agent will forget your most important rules at the worst possible time.

• Close the feedback loop. Hooks without metrics are just guardrails. Hooks WITH metrics are a self-improving system.

AI coding agents are powerful. But power without guardrails is just chaos with better autocomplete.

Build the guardrails. Measure the guardrails. Let the guardrails measure themselves.

That's how you go from "AI writes code for me" to "AI writes code the way I want it to."

🔧 Building this at OrbitxPay — where we're wiring traditional banking to onchain infrastructure. If you're building with AI agents in production, I'd love to hear what guardrails you've put in place.

#ClaudeCode #AIAgents #DevTooling #BuildingInPublic #EngineeringLeadership #Fintech

Why auto-incrementing integers fail at scale

Sequential integer IDs require a centralized counter — a single document or service that every node must consult before inserting. In a distributed MongoDB cluster, this creates three compounding problems. First, every insert needs a network round-trip to the counter authority, adding latency proportional to geographic distance. Second, that counter becomes a single point of failure: if it goes down, no node can insert anything. Third, concurrent access to the counter creates race conditions. MongoDB's own engineering blog illustrates this: two threads read the current value (41), both compute 42, and both attempt to store documents with the same ID — causing a duplicate key error or a lost write. Solving this with atomic findAndModify operations serializes all inserts through a single bottleneck, destroying throughput.

ObjectID sidesteps all of this. The MongoDB driver generates the ID on the application server before the insert ever reaches the database. No coordination, no contention, no central authority. Each client independently produces unique IDs, and insertion throughput scales linearly with the number of application servers.

The 12-byte anatomy of an ObjectID

An ObjectID is 12 bytes rendered as a 24-character hexadecimal string — for example, 507f1f77bcf86cd799439011. Since a 2018 specification revision (formalized for MongoDB 3.4+), those 12 bytes break down into three segments:

Bytes 0–3: Unix timestamp (4 bytes). A 32-bit unsigned integer representing seconds since January 1, 1970 UTC, stored in big-endian byte order. This gives one-second resolution and extends to approximately the year 2106, avoiding the signed 32-bit overflow problem of 2038. Big-endian encoding is deliberate — it allows raw memcmp byte comparison to produce correct chronological ordering, which is critical for index efficiency.

Bytes 4–8: Per-process random value (5 bytes). Generated once when the driver process starts and held constant for the lifetime of that process. This value does not need to be cryptographically secure; a standard PRNG seeded with OS entropy suffices. The 5-byte width provides ~1.1 trillion possible values, making cross-process collisions statistically negligible.

Bytes 9–11: Incrementing counter (3 bytes). Initialized to a random value at driver startup and incremented by one for each ObjectID generated. Stored in big-endian order. The 3-byte counter supports 16,777,216 unique ObjectIDs per process per second before wrapping to zero.

The pre-3.4 format split bytes 4–8 differently: 3 bytes for an MD5 hash of the machine hostname plus 2 bytes for the process ID. MongoDB retired this layout because MD5 cannot be used on FIPS-compliant systems, virtual machines cloned from identical images produced identical machine hashes, and different language drivers implemented these fields inconsistently. Merging machine ID and process ID into a single 5-byte random value solved all three problems while maintaining uniqueness guarantees.

Why ObjectID excels in distributed architectures

The three-segment design creates a layered uniqueness guarantee without any distributed consensus. Documents created in different seconds differ in the timestamp. Documents from different processes in the same second differ in the random value. Documents from the same process in the same second differ in the counter. The only theoretical collision requires a single process to generate more than 16.7 million ObjectIDs within one second — an extreme edge case.

Beyond uniqueness, ObjectID provides roughly monotonic ordering because the leading bytes are a timestamp. Sorting by _id approximates sorting by creation time, and range queries on _id can efficiently select documents from a time window. The embedded timestamp also eliminates the need for a separate createdAt field in many applications — any ObjectID's creation time is extractable via ObjectId.getTimestamp() with one-second precision.

At 12 bytes, ObjectID is more compact than a UUID's 16 bytes (33% smaller) or a string UUID's 36 bytes. Since every MongoDB document carries an _id field with a mandatory unique index, this size difference compounds across millions of documents — affecting index memory footprint, disk usage, and WiredTiger cache efficiency.

Two caveats apply. ObjectIDs created within the same second from different machines have no guaranteed ordering, and clients with skewed system clocks can produce out-of-order timestamps.

Performance costs of random IDs versus monotonic ones

The performance gap between ObjectID and random identifiers like UUIDv4 stems from B-tree index mechanics in WiredTiger, MongoDB's default storage engine. ObjectID's roughly monotonic nature means new entries append to the right edge of the B-tree, keeping the hot working set confined to a few rightmost leaf pages. Only these pages need to reside in the WiredTiger cache for write-heavy workloads.

Random UUIDs scatter inserts across the entire B-tree. Each random insert may land on a page not currently in cache, forcing a disk read. When a leaf page fills, it splits — and random UUIDs cause 500× more page splits than sequential IDs (roughly 5,000–10,000 splits per million records versus 10–20). Each split triggers cascading rebalancing up the tree, producing 5–10× write amplification per logical insert.

Benchmarks confirm the theory. A Go benchmark against MongoDB 6.0.5 with 1 million batched inserts (batch size 10,000) measured ObjectID at 3.32 seconds, ULID at 3.52 seconds, and UUID at 5.16 seconds — 55% slower. The gap widens dramatically at scale: inserting 10 million documents into a collection already containing 10 million took ObjectID about 32 seconds versus over 63 seconds for UUID. Separate benchmarks report UUIDs sustaining roughly 2,000 inserts per second at 10–20 million documents where ObjectIDs sustain 7,500 — a 3.75× throughput difference.

Index size also matters. ObjectID occupies 12 bytes per key; a binary UUID takes 16 bytes (33% more); a string UUID takes 36 bytes (3× more). With MongoDB's typical 4 KB index pages, ObjectID fits approximately 317 entries per page versus fewer for larger key types, directly affecting how much of the index the cache can hold.

Custom IDs are fully supported and sometimes preferable

MongoDB's _id field accepts any BSON data type except arrays, regex, or undefined — strings, integers, UUIDs, dates, embedded documents, and more. To use a custom ID, simply include _id in the document at insert time. If omitted, the driver auto-generates an ObjectID.

The strongest case for custom IDs is natural keys that will never change. Using an email address, username, or external system identifier as _id eliminates the need for a separate unique index on that field, saving both storage and write overhead. MongoDB's documentation explicitly recommends this: "Use a natural unique identifier, if available. This saves space and avoids additional indexes."

Compound _id values — embedded documents like { region: "US", order: 12345 } — work well for data with natural composite keys. In application code, the implementation is straightforward across drivers:

// Node.js
await collection.insertOne({ _id: "user@example.com", name: "Alice" });

// Mongoose schema with custom _id
const schema = new mongoose.Schema({
  _id: { type: String, required: true },
  name: String
});

# PyMongo
collection.insert_one({ '_id': 'user@example.com', 'name': 'Alice' })

MongoDB's Node.js driver also supports a pkFactory option that auto-generates custom IDs when none is provided, enabling application-wide ID strategies without modifying every insert call.

Navigating the trade-offs between ObjectID and alternatives

The decision framework rests on five dimensions, each favoring different ID strategies.

Write performance and scalability strongly favor monotonic IDs. ObjectID and time-ordered alternatives (UUIDv7, ULID, KSUID) maintain right-edge B-tree appends. Random UUIDs (v4) degrade catastrophically beyond 10 million documents. For write-heavy workloads at scale, this is often the deciding factor.

Privacy and security favor random UUIDs. ObjectID's first four bytes encode the creation timestamp in a trivially decodable format. Exposing ObjectIDs in URLs or APIs reveals when every document was created, enabling enumeration attacks and leaking usage patterns. UUIDv4 reveals nothing. UUIDv7 encodes time but in a less immediately obvious format.

Portability across databases favors UUIDs. ObjectID is MongoDB-specific. Applications that might migrate to PostgreSQL, MySQL, or another database benefit from the universal UUID standard. MongoDB's own blog acknowledges this: "Some businesses may be reluctant to link their application logic to an identifier generated by a specific database product."

Storage efficiency favors ObjectID (12 bytes) over binary UUID (16 bytes) and dramatically over string UUID (36 bytes). For collections with hundreds of millions of documents, the 33% index size reduction versus binary UUID translates to meaningful memory and disk savings.

Sharding behavior requires special attention regardless of ID type. Monotonically increasing IDs (including ObjectID) used directly as a shard key route all inserts to a single chunk, creating hotspots. The solution is hashed sharding ({ _id: "hashed" }) or compound shard keys. Random UUIDs distribute writes evenly but sacrifice range query efficiency.

Conclusion

ObjectID is MongoDB's default for a reason: it delivers globally unique, roughly time-ordered, compact identifiers without any distributed coordination — the exact properties a horizontally scalable database needs. Its 12-byte structure packs a timestamp, process-unique random value, and counter into a format optimized for B-tree index performance and memcmp sorting.

Custom IDs make sense in specific scenarios — natural keys that eliminate redundant indexes, UUIDs required for cross-database portability, or random IDs needed to prevent timestamp leakage. The critical insight is that ID ordering matters more than ID type: any time-ordered identifier (ObjectID, UUIDv7, ULID) will dramatically outperform random ones at scale due to B-tree mechanics. Applications choosing custom IDs should prioritize monotonic or semi-monotonic strategies, store UUIDs as binary rather than strings, and test write performance at target collection sizes before committing to an ID scheme.

Ever feel like your main job is just being a human reminder? You spend your day asking for updates, checking if a PR was reviewed, or wondering why a task hasn't moved in three days. It’s exhausting for you and, honestly, it’s not great for the team either. 😫

Building a team that "self-delivers" isn't about hiring superheroes. It’s about shifting the culture from "doing tasks" to "owning outcomes." Let’s look at how to stop the constant follow-ups and build a high-agency environment.

Understanding the Agency Gap

For a junior engineer or someone new to the field, work often feels like a checklist. You get a ticket, you write the code, you move it to "Testing," and you're done. This is low agency. The mindset is: "I did my part, now it's someone else's problem." 🛑

High agency is the exact opposite. It’s the refusal to let a project stall just because a hurdle appeared. If a high-agency engineer sees a delay, they don't just wait; they find a way around it or pull the right people together to fix it. They don't just own the code; they own the result.

The "Not My Problem" Trap

We’ve all heard it: "It’s pending on DevOps" or "I’m waiting for the designer." When people say this and then sit idle, the momentum dies. To fix this, you have to reward the behavior of clearing blockers, not just finishing tickets.

How to Build a Culture of Ownership

Ownership can't be forced, but it can be engineered. You have to create an environment where taking initiative is the path of least resistance. 🛠️

1. Define the 'Definition of Done' clearly. Done doesn't mean the code is pushed. Done means it's in the hands of the user and working as expected.
2. Stop the micro-management. If you follow up every hour, the team learns they don't need to track their own progress because you'll do it for them. Step back and see if they pick up the slack.
3. Give them the 'Why,' not just the 'How.' When engineers understand the business impact of a feature, they’re more likely to care about it actually crossing the finish line.

Removing the Hurdles (The Process Fix)

Sometimes, the team isn't lazy; the process is just broken. If your workflow requires five manual approvals and a blood sacrifice to deploy, people will naturally lose interest. 📉

Manual approvals are momentum killers. If someone has to wait for a manager to click a button, they'll check out and start something else. By the time the approval comes, they've lost their flow.

- Automate everything possible. Use automated testing and CI/CD to replace manual sign-offs.
- Empower peer reviews. Let the team trust each other rather than waiting for a single "gatekeeper."
- Reduce handoffs. Every time a task moves from one person to another, there's a 50% chance it gets dropped. Keep the owner the same from start to finish if you can.

Spotting Burnout vs. Boredom

When a high-performer starts saying "I've done my stuff, it's pending elsewhere," it might be a red flag for burnout. Burnout doesn't always look like exhaustion; sometimes it looks like cynicism or apathy. 🚩

If someone feels like their effort doesn't matter because the system is too slow, they stop trying. They check out emotionally to protect themselves from the frustration of being blocked.

How to spot it:

- The 'Silent' Developer: Someone who used to be vocal in meetings but now just nods.
- The Bare Minimum: They do exactly what's on the ticket, but nothing more—even if there's an obvious bug right next to their change.
- Frequent Friction: They seem annoyed by small requests or process changes.

Key Takeaways for Leaders

To fix a team that needs constant follow-ups, you have to stop being the one who follows up. Shift the responsibility.

- Make work visible. Use a board where blockers are glaringly obvious so the team feels the pressure to clear them, not you.
- Kill the wait times. Audit your workflow this week. Find one manual approval and delete it. See what happens.
- Celebrate the 'Finishers.' Don't just praise the person who wrote the most code. Praise the person who chased down the stakeholder to get the final sign-off. 🏆

High agency is contagious. Once a few people start taking real ownership, the rest of the team usually steps up to match that energy.

#Leadership #Engineering #Productivity

#EngineeringLeadership #HighAgency #TeamCulture

Imagine you and your two best friends are keeping score in a video game. One friend is the "scorekeeper" and announces all the points. But what if the scorekeeper's controller breaks mid-game? You need a way to agree on the score without them.

Viewstamped Replication is like having a protocol: "If the scorekeeper fails, we pick the next person in a predetermined order—no arguing." You also write down every score on paper (all three of you), so even if someone forgets, you can check the paper and catch up.

Abstract

Viewstamped Replication (VSR) is a consensus algorithm that keeps replicated systems consistent when nodes fail. Here's how it works:

Key Concepts

1. Normal Operation: A designated "primary" node receives requests from clients, stamps them with a view number (like a logical clock), and sends them to backup replicas. The primary waits for acknowledgment from a quorum (majority) before committing the operation.

2. View Change (Failover): If the primary becomes unresponsive (detected via timeout), the system automatically transitions to a new view. The next replica in a deterministic round-robin order becomes the new primary—no elections, no voting.

3. Quorum Intersection: Every committed operation is known by at least f+1 replicas in a 2f+1 system. When a new primary takes over, it queries at least f+1 replicas to find the latest committed state, ensuring no data loss.

4. View Numbers: Operations are tagged with view numbers, ensuring ordering even across view changes. If two operations have the same sequence number, the one from a higher view number "wins."

The beauty: it's simpler than Raft (no split-vote elections) and faster (deterministic leader selection, parallel replication).

VSR Normal Operation Flow

sequenceDiagram
    participant Client
    participant Primary
    participant Replica1
    participant Replica2

    Client->>Primary: Request (view=5, seq=100)
    Primary->>Primary: Stamp request
    Primary->>Replica1: PREPARE (view=5, seq=100, op)
    Primary->>Replica2: PREPARE (view=5, seq=100, op)
    Replica1->>Replica1: Log entry
    Replica2->>Replica2: Log entry
    Replica1->>Primary: PREPAREOK
    Replica2->>Primary: PREPAREOK
    Primary->>Primary: Quorum acknowledged
    Primary->>Replica1: COMMIT (view=5, seq=100)
    Primary->>Replica2: COMMIT (view=5, seq=100)
    Replica1->>Replica1: Execute operation
    Replica2->>Replica2: Execute operation
    Primary->>Client: ACK (result)

VSR View Change Flow

sequenceDiagram
    participant OldPrimary as Old Primary<br/>(Failed)
    participant Replica1
    participant Replica2
    participant NewPrimary as New Primary<br/>(Replica1)

    Replica2->>Replica2: Heartbeat timeout
    Replica2->>Replica1: START-VIEW-CHANGE (view=6)
    Replica2->>Replica2: START-VIEW-CHANGE (view=6)
    Replica1->>Replica1: Receive START-VIEW-CHANGE
    Replica1->>Replica1: Advance view to 6
    Replica2->>Replica1: DO-VIEW-CHANGE (view=6, log_info)
    Replica2->>Replica2: DO-VIEW-CHANGE (view=6, log_info)
    Replica1->>Replica1: Quorum collected
    Replica1->>Replica2: START-VIEW (view=6, log)
    Replica1->>Replica2: Become New Primary
    Replica2->>Replica2: Accept view 6
    NewPrimary->>Replica2: Normal operation resumes

Architecture

VSR achieves strong safety guarantees with superior performance characteristics compared to Multi-Paxos and Raft. Here's the technical depth:

Key Architectural Differences from Raft

Deterministic Leader Selection

• Unlike Raft's randomized election timeouts (which introduce latency variance), VSR pre-determines the next primary as (current_primary_index + 1) mod cluster_size. This eliminates split-vote scenarios and achieves sub-millisecond failover.

Passive vs. Active Replication

• Raft employs active replication—all replicas independently execute operations. VSR uses passive replication—only the primary executes operations, reducing state machine redundancy overhead. Backups maintain logs purely for recovery.

Recovery Protocol

• VSR includes a built-in recovery protocol for crashed nodes rejoining the cluster. A recovering node requests state from at least f+1 replicas (including the current primary) and applies the latest known committed state. This decouples recovery from durable storage guarantees, unlike Raft which assumes perfect storage.

View Change Protocol

• When a replica detects primary failure (via heartbeat timeout or explicit view-change messages), it advances its view number and broadcasts START-VIEW-CHANGE. When a majority of DO-VIEW-CHANGE messages is collected, the designated new primary sends START-VIEW with the highest-numbered log from the quorum, ensuring linearizability.

Comparison Table: VSR vs Raft vs Paxos

Quorum Intersection Property

graph TD
    A["Cluster: 5 Nodes<br/>(N=5, f=2)"] --> B["Quorum Size: 3"]
    B --> C["Any two quorums<br/>must overlap by ≥1"]
    C --> D["Primary commits<br/>with 3 replicas"]
    D --> E["New primary queries<br/>f+1=3 replicas"]
    E --> F["Guaranteed to find<br/>latest committed state"]

VSR State Machine Architecture

stateDiagram-v2
    [*] --> NormalMode

    NormalMode --> ViewChange: Primary fails<br/>or heartbeat timeout
    NormalMode --> NormalMode: Client requests<br/>PREPARE + COMMIT

    ViewChange --> DetermineLead: Elect new primary<br/>via round-robin
    DetermineLead --> SyncState: New primary queries<br/>f+1 replicas
    SyncState --> NormalMode: Log synchronized<br/>Ready to serve

    ViewChange --> ViewChange: If new primary fails

How VSR Powers TigerBeetle's 1000x Throughput

Throughput Optimization Flow

graph LR
    A["Transaction Batch<br/>8,000 txns"] --> B["Stamp with<br/>View Number<br/>O1"]
    B --> C["Send to Quorum<br/>in Parallel"]
    C --> D["Replicas Log<br/>in Parallel"]
    D --> E["Quorum Ack<br/>f+1 nodes"]
    E --> F["Execute Serially<br/>on State Machine"]
    F --> G["Commit + Response<br/>to Clients"]

    style A fill:#e1f5ff
    style G fill:#c8e6c9

Why VSR Wins for Financial Ledgers

1. Batching Efficiency: TigerBeetle batches up to 8,000 transactions per batch. VSR's view stamping (a lightweight logical clock) costs almost nothing—just an integer increment and replication.

2. Normal Path Optimization: In the steady state, VSR consensus isn't engaged at all—replicas simply append to their logs in parallel and execute serially on the replicated state machine. This bypasses heavy consensus coordination.

3. Deterministic Failover: No leader election storms. When the primary fails, the next replica knows immediately it's next (round-robin). No heartbeat storms, no cascade of election messages—just clean handoff.

4. Quorum Replication: Requiring f+1 acknowledgments before commit (not 2f+1) means TigerBeetle replicates to a majority faster than protocols requiring unanimous replication, reducing tail latency.

5. Immutable Ledger Integration: VSR's append-only nature aligns perfectly with financial ledger semantics. Every transfer is stamped with a view number that serves as a global ordering—this becomes your audit trail automatically.

Performance Comparison

graph LR
    A["VSR<br/>Batching"] -->|8K txns| B["1 Stamp<br/>1 Replication<br/>1 Execution"]
    C["Raft<br/>Per-txn"] -->|1 txn| D["1 Election check<br/>1 Replication<br/>1 Execution"]

    B --> E["~1000x more<br/>throughput"]
    D --> E

    style E fill:#fff9c4

Key Advantages of VSR

Safety Guarantees

graph TD
    A["Linearizability"] --> B["Every operation appears<br/>atomic"]
    C["Durability"] --> B
    D["Consistency"] --> B
    B --> E["No data loss<br/>No double-spend"]
    E --> F["Financial Guarantee"]

    style F fill:#c8e6c9

Performance Under Load

Why This Matters for Financial Systems

1. Predictability: Deterministic failover means SLAs are achievable. No random election storms.

2. Throughput: Financial transactions at scale demand near-database speeds. VSR delivers.

3. Correctness by Design: Debit/credit invariants + VSR consensus = no double-spend bugs.

4. Operational Simplicity: One binary, no external coordinators. Reduces failure modes in production.

5. Audit Trail: Every transaction stamped with a global order. Compliance-friendly by default.

Summary

Raft became famous because it's understandable. But understandability shouldn't be the only metric—availability and performance matter too. VSR predates Paxos (1988) and solved these problems decades ago. TigerBeetle resurrects it for a new era where financial systems demand both correctness at scale and throughput that doesn't compromise safety.

If you're building systems where a millisecond of failover latency or a percentage point of throughput matters, VSR isn't just academic—it's a pragmatic engineering choice.

Key Takeaways

• VSR = Deterministic + Passive + Efficient

• Raft = Democratic + Active + Understandable

• For Finance: VSR wins on latency, throughput, and predictability

• TigerBeetle proves VSR is production-ready at scale

Imagine you're managing a very popular restaurant. You need to know two things:

• How many different people visited today?

• How many times did each person order food?

There two clever tools: HyperLogLog (HLL) and Count-Min Sketch.

HyperLogLog (HLL) - The Unique Visitor Counter

Think of HLL like a magical guest book that:

• Only needs a tiny amount of space (1.5KB)

• Can estimate millions of unique visitors

• Might be off by 2% (which is totally fine!)

graph TD
    A["Visitors"] --> B["Magic Guest Book (HLL)"]
    B --> C["Approximate Count"]
    style B fill:#f9f,stroke:#333

Count-Min Sketch - The Smart Traffic Counter

This is like having multiple security cameras counting how often each person comes in:

• Each camera counts a bit differently

• By combining all cameras' counts, we get a good estimate

• Catches people who are visiting too frequently

graph LR
    A["Requests"] --> B["Multiple Counters"]
    B --> C["Counter 1"]
    B --> D["Counter 2"]
    B --> E["Counter 3"]
    C & D & E --> F["Final Count"]

The Technical Deep-Dive

Now let's look under the hood at how these systems actually work.

HyperLogLog Implementation

class HyperLogLog {
    constructor(precision = 14) {
        this.precision = precision;
        this.m = 1 << precision; // Number of registers
        this.registers = new Array(this.m).fill(0);
    }

    add(element) {
        const hash = this.hashFunction(element);
        const bucket = hash & (this.m - 1); // Determine register
        const w = hash >>> this.precision;
        this.registers[bucket] = Math.max(
            this.registers[bucket],
            this.leadingZeros(w) + 1
        );
    }

    estimate() {
        // Harmonic mean calculation for cardinality estimation
        const harmonicMean = this.registers.reduce((sum, val) => 
            sum + Math.pow(2, -val), 0);
        return (this.m * this.m) / harmonicMean;
    }
}

Count-Min Sketch Technical Details

class CountMinSketch {
    constructor(width, depth) {
        this.width = width;
        this.depth = depth;
        this.table = Array.from(
            { length: depth },
            () => new Array(width).fill(0)
        );
        this.hashFunctions = this.generateHashFunctions();
    }

    add(item, count = 1) {
        this.hashFunctions.forEach((hashFunc, i) => {
            const j = hashFunc(item) % this.width;
            this.table[i][j] += count;
        });
    }

    estimate(item) {
        return Math.min(
            ...this.hashFunctions.map((hashFunc, i) => 
                this.table[i][hashFunc(item) % this.width]
            )
        );
    }
}

Real-World Implementation at Scale

graph TD
    A["Incoming Request"] --> B["Edge Node"]
    B --> C["HLL Counter"]
    B --> D["Count-Min Sketch"]
    C & D --> E["Redis Cluster"]
    E --> F["Rate Limit Decision"]
    style B fill:#96f,stroke:#333
    style E fill:#f96,stroke:#333

Key Technical Considerations

• Memory Efficiency: Both algorithms use constant memory regardless of traffic volume

• Error Bounds: HLL provides ±2% error rate, Count-Min Sketch error is configurable

• Distributed System: Uses Redis CRDTs for counter synchronization across edge nodes

• Performance: O(1) time complexity for both updates and queries

By combining these probabilistic data structures, we can efficiently track and limit millions of requests per second, all while maintaining minimal memory usage and providing accurate rate limiting capabilities.

Real-World Use Cases

API Management: An API processing 10M requests per day uses CMS to identify abusive clients without tracking individual IPs.
E-commerce: HLL prevents coupon abuse by estimating unique users during flash sales.
Cybersecurity: The combination of both algorithms detects DDoS attacks with minimal memory overhead.

Comparison: Traditional vs. Probabilistic Approach

When To Avoid This Approach

• Banking transaction systems requiring exact counts

• Legal compliance scenarios needing 100% accuracy

• Systems with burst-spike allowances

References:

https://redis.io/docs/latest/develop/data-types/probabilistic

Key Characteristics of CRDT-Friendly Systems

1. Decentralized Updates

   • Allow independent modifications across distributed replicas (e.g., collaborative text editors, IoT sensor networks).

   • Example: Multiple users editing the same document offline.

2. Mergeable Operations

   • Use commutative, associative, and idempotent operations to ensure conflict resolution.

   • Example: Incrementing counters (counter += 1) or adding elements to a set.

3. Eventual Consistency

   • Prioritize availability over strong consistency, tolerating temporary divergences (e.g., social media "likes" syncing across regions).

4. Scalable State Management

   • Optimize for minimal data transfer (e.g., delta-CRDTs sending only recent changes instead of full state)

CRDT-Friendly Data Structures and Use Cases

Systems That Benefit from CRDTs

• Distributed databases (e.g., Redis, Riak) for conflict-free replication.

• Collaborative apps (e.g., Figma, Apple Notes) supporting real-time sync.

• IoT networks handling offline data collection and delayed merges.

Non-CRDT-Friendly Scenarios

• Strong consistency requirements (e.g., banking transactions).

• Complex operations needing intent tracking (e.g., rich text formatting in TinyMCE).

CRDT-friendly designs simplify distributed systems by eliminating consensus protocols, making them ideal for high-availability, low-latency applications

Blockchain and its Basics

Blockchain, is a combination of a decentralized and distributed database containing a registry of transactions that are distributed among peers or fellow participants in the network. The registry includes a long list of transactions and is continually updated with new transactions as they take place. Starting from the very first transaction, a bunch of transactions are grouped into a block as per a predefined block size (1 MB in case of Bitcoin). Once the block size is achieved by one block, the next set of transactions forms another block which is then linked to the block previously formed. Over time, a series of blocks is formed where each block is connected to another block that was created just before it. Thus, we call this chain of blocks the blockchain.

In the existing world, newspapers can be a loose example of where every piece of news is a transaction, Every day its new block and printing press is a block producer and every reader is a node.

To create a basic blockchain, following existing technologies and techniques were used.

1. Digital Signatures
2. Hashing
3. Merkle Trees
4. Hash Cash
5. TCP/IP
6. P2P Network

Cryptocurrency is a reward for node operators to run a blockchain node to cover their operating costs. Without this public chain can't exist.

Digital Signatures

A digital signature acts as a verifier of the authenticity of the sender.

There are two types of digital signatures:

Symmetric digital signatures:

• One single key is used to encrypt the messages.
• The sender encrypts the message with that key and sends it to the receiver.
• Once the receiver receives it, they need the same key to decrypt or unlock that message.
• So, the sender also shares the key he/she encrypted the message with so that the receiver can use it to decrypt the message.

Asymmetric Digital Signatures:

• A pair of public and private keys are used.
• A message encrypted with a public key can only be decrypted with the corresponding
• Private key of the public-private key pair and vice versa.
• The public key of each participant is shared across the network and the private key is held secret by the owner/sender.

Additionally, in the case of normal web traffic, combination of both Asymmetric and Symmetric has been used to share the information between the browser and the server. During the initial step, Asymmetric Digital Signatures are used to form a secure connection and then A Symmetric Key will be shared between both parties. This will help to reduce the overhead in subsequent traffic.

Hashing

It is an encryption technique which is used to encrypt the data to ensure data security, and it's done using hash functions. The hash function is a mathematical function that can take in any length of input and convert it to an output of fixed length. The output hash serves like a fingerprint for that data.

Hash uniquely represents that data. Even a small change in the data generates an entirely random and different hash.

Hashing & its uses

This technique is used to pack transactions into block. So, if a bad actor tries to tamper with the existing transaction, it will result in hash output mismatch.

Merkle Tree

Merkel Tree is a way of organizing transactions in the block. The output of a Merkle tree is a Merkle root. The Merkle tree utilizes hash functions to arrive at the Merkle root by recursive hashing.

Even the smallest of changes in any of the data points will change its hash and hence, the root hash will also be changed. Thus, root hash is the fingerprint representing all the data points or block.

The Merkle trees help in verifying transaction data on a block by acting as a unique identifier for the transactions.

Hash Cash

Hash cash is a proof of work the sender of an email must do before sending an email. The sender must calculate the hash of the email data and make sure that the calculated hash satisfies a predefined condition. This limits the scope of spam emails.

The technique of hash cash is used in the process of creation of blocks in the blockchain. So that miner nodes are spending their resources (Computational power, memory, electricity etc.) to form a block.

TCP/IP

The TCP/IP is a communication protocol which sends data in the form of packets over a server using the IP addresses of the sender and receiver.

Peer to Peer Network

In peer-to-peer (P2P) networking, a group of computers are linked together with equal permissions and responsibilities for processing data. Unlike traditional client-server networking, no devices in a P2P network are designated solely to serve or to receive data.

In a peer-to-peer network, each device can act as a server and as a client at different points in time. In a blockchain peer-to-peer network, the peers use TCP/IP protocols to connect to other peers and transfer the data.

Basic Architecture

All the above-mentioned techniques to create the ideal business network, i.e., blockchain. The architecture of the blockchain network is as follows:

• There is a peer-to-peer network created between all the participants of the network.
• Each network participant has a digital signature for identifying the participants Hash Cash TCP/IP and Peer to Peer Network Basic Architecture of Blockchain
• Any transaction happening between two network participants gets flooded in the network with the help of gossip protocol to all the participants for validation.
• Each participant has a transaction pool which is a memory space allocated to store the verified transactions.
• Each transaction is secured using the hashing algorithm.
• The transaction pool is at a node level.
• There are designated nodes which create the blocks out of the transactions happening in the network at a frequency known as mining rate.
• Once the block is created it gets flooded in the network and each network participant verifies the block of data and once there is a single source of truth for the block, the block gets added to the blockchain.

Consensus

Since public blockchain is an open source network, there should be a way for participants to mutually agree on the transactions and blocks in blockchain. So there is a need for a consensus layer.

Mathematical algorithms are used to verify transactions and ensure trust between transacting parties. Transacting parties have to trust the output achieved using these mathematical algorithms.

These algorithms together are known as the consensus mechanism in the blockchain. The consensus can for a transaction or for an entire block

There are many types of consensus algorithms; proof of work, proof of stake, practical BFT etc. Every blockchain network has formed its own consensus protocol based on their network needs.

Data Immutability

Blockchain offers one more important feature that differentiates it from other networks: Data Immutability.

In practical terms, it refers to the extreme difficulty that one will face in trying to alter or make changes to the existing data.

Blockchain offers data immutability as follows:

• Blockchain forms a chain of blocks which are connected to each other via a link, also known as the previous hash pointer or simply the previous block hash.
• The hash of a block is calculated for all the contents in the block header using a hashing algorithm.
• Merkel root is a part of the block header and any change in any constituent transaction results in the change in the Merkel root.
• Change in the Merkel root will result in changing the hash of the entire block.
• If the hash of one block is changed, the block next to it will not have a link to this block as the previous hash will not match the new block.
• Hence, the link between the blocks will be broken, and these blocks will become invalidated.

Immutability is an important feature of the blockchain which helps protect data from being manipulated and also helps in identifying malicious nodes in the network.

Block header hash = SHA256 (previous block hash + Merkle root + timestamp + difficulty target + nonce) .

*Challenges in protecting a blockchain network

51% attack: In case more than 51% of the nodes in the network are malicious the network could become unstable.*

BFT Network

For a network to be byzantine fault tolerant the number of malicious nodes in the network should be less than 1/3rd of the total nodes in the network. Whenever a node receives two conflicting messages, it goes for a majority vote and accepts the message which comes from majority number of nodes. To ensure that the correct message is accepted by the nodes in the network the total number of malicious nodes needs to less than 33.33% or 1/3rd of the network

(3F+1) - Where is f is no of faulty nodes that can be present in the network

Types of Blockchain Networks

1. Public - Bitcoin, Ethereum
   • Permissionless: A public permissionless blockchain is free for anyone to join or leave. This type of network provides anonymity, immutability and transparency but compromises on efficiency.Public
     • Permissioned: A public permissioned blockchain is an intermediate between private and public networks. It values efficiency and immutability over transparency and anonymity, where every participating member is aware of the identities of the other members in the network.
2. Private / Enterprise - Hyperledger Fabric, Hyperledger Besu
   • A private blockchain is one which is operated and managed by a single entity. These type of blockchains are generally applicable in the case of a conglomerate where the parent company runs the network for the underlying group of companies. In such a situation, they value efficiency over anonymity, transparency and immutability.
3. Consortium
   • A consortium blockchain is largely similar to a private blockchain but differs when you consider who controls or manages the network. Instead of concentrating all power in one entity, authority is distributed across two or more participants.
     1. Hybrid (Public + Private)
   • It is used by some private solution providers where they roll up the transactions on their private chain and submit them to a public blockchain for integrity and data immutability.

Bitcoin

It's a first generation Blockchain with a vision of open money.

Ethereum

Initially, the blockchain was used only for the transfer of values between multiple parties. Later, the blockchain community saw an opportunity to make advancements and that gave birth to smart contracts.

Smart Contracts

A smart contract is the digital version of an agreement contract that runs on the Blockchain nodes and is executed when a set of trigger criteria are met. If designed right, a smart contract will surely be successful as it is driven by a computer system without human intervention.

The Smart contract has evolved through the following phases:

• State Machine: The very first version of the blockchain was a state machine. It was just like an accounting ledger, where it held only the state of the node.
• Smart Contracts: All the nodes within a computer network have a code associated with these nodes, which can run on its own based on predefined logic known as smart contracts.
• Oracles: Oracles are external agents which connect the smart contract to the external world. Any external data that is required for the smart contract will be fetched by agents called oracles.

Introduction

Ethereum is a decentralised, open-source, distributed computing platform that enables the creation of smart contracts and decentralised applications, also known as Dapps.

It is essentially a transaction-based state machine. State machine refers to something that would read from a series of inputs and then transition to a new state based on those inputs.

The Ethereum blockchain starts from the ‘genesis state’, which is the first state. And after every transaction, the state of the entire blockchain changes.

https://ethereum.org/en/whitepaper/

Basics

Ether

• Ether is the main currency on the Ethereum blockchain.
• It is a form of payment made by the client of the platform to the machine that executes its requested operation.
• Every time a block is validated, 5 Ethers are created and awarded to the successful node.
• 1 Ether = 1018 wei (Smallest Unit)
• 1 Gwei = 109 wei

Gas

• Gas is a unit to measure the fee required for a computation.
• It refers to the fee required to successfully perform a transaction or execute a contract on the Ethereum blockchain
• The exact price of gas is determined by the network’s miners, who can decline from processing a transaction if the gas price does not meet their threshold.

Gas Price

• Gas price is the amount of Ether paid per unit of Gas spent by a miner for running a transaction

Gas Limit

Gas limit is the maximum amount of Gas required to run a transaction.

Transaction limit is the maximum amount that the sender would need to spend in order to perform a transaction successfully.

Transaction Limit = Gas Limit X Gas Price

Gas price and Gas limit are set by the sender of a transaction.

The system would deduct this amount from the sender based on the amount of Gas consumed in running the transaction.

Accounts

In Ethereum, the state is made up of objects called "accounts", with each account having a 20-byte address and state transitions being direct transfers of value and information between accounts. An Ethereum account contains four fields:

• The nonce, a counter used to make sure each transaction can only be processed once
• The account's current ether balance
• The account's contract code, if present
• The account's storage (empty by default)

Accounts in Ethereum are stored in a global ‘shared state’ in the form of key–value pairs.

The list of all these key–value pairs representing accounts defines the state of Ethereum at that point.

Ethereum has two types of accounts:

1. Externally owned accounts (EOAs)
2. Contract accounts (CAs).

Externally Owned Accounts (EOAs):

These are combinations of public addresses and private keys, and there is no code associated with them.

• Send and receive Ether to/from another account,
• Send transactions to smart contracts.

Contract Accounts (CAs):

These accounts do not have a corresponding private key. These accounts are generated when you deploy your contract on blockchain. You will see them referred to as just contracts

• Send and receive Ether just like EOAs.
• Unlike EOAs, they have code associated with them.
• Transactions have to be triggered by an EOA or another contract.

In a key–value pair, the key is a 20-byte string, which is usually the public address of the account.

Nonce: For EOAs, nonce is the number of transactions that are sent from an account’s address. For CAs, nonce is the number of contracts created by the account.

Balance: It is calculated as Number of wei/Ether owned by an account.

Storage Root (Storage Hash): This is a hash of the root node of a Merkle tree that encodes the hash of the storage content of an account.

Code Hash: This is the hash of the EVM (Ethereum Virtual Machine) code of this account

State Root

Whenever a new block is created, a state root is stored in the header of that block. State root is the Merkle root of all the accounts at that moment. Simply put, all the key–value pairs that represent an account together, form a Merkle tree is known as state root.

This state root is captured by a block at the time of its creation. Any change in the data would lead to the calculation of the Merkle tree all over again to match the state root in the block.

This is highly impossible, and hence the immutability is maintained in the network. Along with state root, the transaction root and the receipt root are also used to capture the state of the network in every block. By calculating and storing the roots mentioned above, Ethereum captures the network state every time a new block is created (state root, transaction root and receipt root).

Transactions

Interaction between accounts are called transactions. Its a signed data package that stores a message that is to be sent from an EOA to another account on the blockchain.

The following points summaries the transactions in Ethereum:

• An EOA can send messages to other EOAs or to other CAs by creating and signing a transaction using its private key.
• A message between two EOAs is simply a value transfer.
• But a message from an EOA to a CA activates the CA’s code, allowing it to perform various actions (e.g., transfer tokens, write to internal storage, mint new tokens, perform some calculation, create new contracts, etc.).
• Unlike EOAs, CAs cannot initiate new transactions on their own.
• Instead, CAs can only fire transactions in response to other transactions that they have received (from an EOA or from another CA).
• An action that occurs on the Ethereum blockchain is always set in motion by transactions that are fired from EOAs.

Note: Transactions can be triggered from an EOA only. Components of a Transaction Nonce: A count of the number of transactions sent by the sender

Gas Price: The amount of Wei that the sender is willing to pay per unit of Gas required to execute the transaction

Gas Limit: The maximum amount of gas that the sender is willing to pay to execute a transaction

to: The address of the recipient. In a contract-creating transaction, an empty value is used

value: The amount of Wei to be transferred from the sender to the recipient. In a contract-creating transaction, this value serves as the starting balance within the newly created contract account

v, r, s: Used to generate the signature that identifies the sender of the transaction

init: An EVM code fragment that is used to initialise the new contract account

data: The input data (i.e., parameters) of the message call. Each type of transaction has all the components above Types of Transaction

Message Calls: These are internal transactions between an EOA and a CA or between one CA and another. When one CA sends a message to another CA, the associated code that exists on the recipient CA is executed.

Value Transfer: These are transfer of value from one EOA to another.

Contract Creation Calls: They are initiated by EOAs, and the recipient’s address is kept empty. Such transactions create a new CA and, hence, are used to create and install new Ethereum contracts

Block

Blocks are batches of transactions with a hash of a previous block in the chain

A block consists of

• Header
• List of Transactions
• Headers Hash of Ommer Block
• Mining details

Consensus in Ethereum

Ethereum is currently operating on a Proof-of-Work (PoW) consensus protocol, but in the future, it will be shifting to a Proof-of-Stake (PoS) protocol.

The current Ethereum blockchain uses a consensus algorithm called Ethash, which is built specifically for the Ethereum blockchain.

The Ethash PoW algorithm introduces the property of ‘memory hardness’ to the Ethereum blockchain.

Ethereum made its mining process highly I/O intensive. This is done with the help of a DAG (directed acyclic graph) – a very large file that is passed along with a block as an input to the hashing algorithm, Ethash.

The DAG requires sufficient memory size, and so, the entire hashing process is a CPU-based rather than a GPU-based process.

Therefore, any system with memory large enough to hold a DAG can now stand a chance to mine the block successfully. And this is why Ethereum has a 15-second block creation rate, as compared with 10 minutes for Bitcoin.

Forks Forks can be classified as accidental or intentional.

Soft Fork

An accidental fork is created when two or more miners find a block at nearly the same time.

The fork is resolved when subsequent block(s) are added and one of the chains becomes longer than the alternative chain(s).

The network abandons the blocks that are not in the longest chain (they are called orphaned blocks).

Ommer Block Ommer/uncle blocks are created in Ethereum blockchains when two blocks are mined and submitted to the ledger at roughly the same time. Only one can enter the ledger as a block, while the other does not. They are similar to Bitcoin orphans but have an integrated use, unlike their Bitcoin counterparts. The orphan block is called the Ommer block

Hard Fork

A hard fork is a permanent split of a blockchain, and it is backward-incompatible.

It divides a blockchain into two separate chains, both of which are dominant.

A hard fork is generally a change in protocol that renders the older version invalid.

Ex: Ethereum and Ethereum Classic

Transaction Flow

Logs and Receipts

Ethereum maintains logs to track various transactions and messages.

A contract can explicitly generate a log by defining events.

A log entry contains:

1.The account address,

1. Events
2. Data.

A log stored in the header comes from the log information present in the transaction receipt.

Ethereum generates a receipt for every transaction.

These receipts include:

1. Block number,
2. Block hash,
3. Transaction hash,
4. Gas used,
5. Cumulative gas used in the block
6. Logs of current transaction

State Transition

Ethereum is a State Transition Machine, which is a process that takes the system from an existing state (N1) to a new state (N2) after a transaction is given as an input to the network.

Ethereum keeps moving between these states as new transactions keep occurring on the network.

Whenever a transaction is triggered by an EOA, that transaction has to go through certain checks and a specific process as follows

1. valid signature
2. valid transaction / sender account nonce
3. gas limit
4. sender account balance

Once all parameters are valid, the transaction gets executed as follows

• Upfront cost of execution is deducted from the sender’s balance
• Nonce of the sender’s account is increased by 1 to account for the current transaction
• When the transaction starts executing. Throughout the execution, Ethereum keeps track of the ‘substate’, which is a way to record information accrued during the transaction that would be needed immediately after the transaction is complete
• A self-destruct set, which is a set of accounts (if any) that would be discarded after the transaction completes
• Log series, which are archived and indexable checkpoints of the virtual machine’s code execution
• Refund balance, which is the amount to be refunded to the sender’s account after the transaction
• The various computations required by the transaction are processed.
• Once all the steps required by the transaction have been processed, and assuming there is no invalid state, the state is finalised by determining the amount of unused gas to be refunded to the sender

Tokens

Ethereum tokens are simply digital assets with standards that are being built on top of the Ethereum blockchain. They benefit from Ethereum’s existing infrastructure instead of developers having to build an entirely new blockchain.

Ethereum is a platform that can be used to create any arbitrary smart contract including smart contracts that represent digital assets called Ethereum tokens.

These token standards are called ERC standards.

ERC stands for Ethereum Request for Comments. These are application-level comments or standards defined for the token. Anyone can create an ERC. However, the ERC’s author must include a clear explanation of its standard.

Applications can use this common ERC token and make it easier to interact with each other. The conditions of the standard can include anything; for example, the way it will interact with a smart contract.

Some of the common ERC standards are ERC 20, ERC 721 and ERC 777 etc. Fungible and Non-Fungible Tokens Fungible tokens

• These are tokens with identical properties.
• One token can be interchanged with another token of the same value.
• Tokens are divisible into smaller amounts as long as they add up to the same
• Some of the examples of fungible token standards are ERC 20, ERC 777 and ERC 223.

ERC - 20

• ERC 20 is an asset created on Ethereum and is written in Solidity.
• It is hosted on the Ethereum blockchain.
• ERC 20 tokens are stored and sent using Ethereum addresses and transactions.
• This standard has six functions – totalSupply, balanceOf, transfer, transferFrom, approve and allowance.
• These functions can be used to transfer tokens to an address or a smart contract.

ERC 777

• It is similar to ERC 20 with extra functionalities like mint and burn.

Non-Fungible Tokens

• These are tokens with unique properties. Even if two tokens are of the same type, they are unique
• No two tokens can be interchanged as they all have unique specifications.
• As these tokens can represent an identity, they are not divisible into smaller units.

ERC - 721

• Every non-fungible token (like ERC 721) has a metadata associated with it which defines the token.
• This metadata is unique for each token. For example, suppose you have a land registry document set. Each document represents one token.
• Creating a unique set of tokens is also known as tokenizing assets on the blockchain network where each document can represent an asset with certain properties

Challenges in public ethereum

• Scalability/throughput: The number of transactions per second is very low
• Cost: The mining process is expensive and transaction gas fees is also very high
• Size: The size of the network is increasing, and nodes have to be highly equipped in order to store a blockchain of this size

Merkle Patricia Trie

Ethereum stores data in both Merkle trees and a radix tree called Patricia trie.

• Ethereum combines the properties of a Merkle tree and a Patricia trie to create a modified Merkle Patricia trie.
• A Merkle tree can be used only to check whether a value is present in the tree or not.
• A Patricia trie (a radix tree) is a data structure that stores values in key–value pairs. This is optimum in finding common prefixes of the searched value.
• These combined properties of both the trees yields a structure that is more optimum for storing and checking values than Merkle tree individual. Ethereum uses a database to store its trie structure. Some examples of such databases could include LevelDb and RocksDB. The purpose of storing this data is to go forward and backward, and identify the exact data that was tampered with.

In the next article, I will write about Applications of Ethereum.

Let's say you have added a backlink to a great website for whatever reason. But for some reason, they closed the shop and that domain was available for sale OR somehow a hacker got control of that domain.

The screenshot of expired domains shows that there are many domains with millions of backlinks to them. If a hacker gets hold of some domain which has a backlink from a legitimate site, and Boom !!

Their users are vulnerable to this kind of hack. Even a simple email ID theft can also cause larger damages.

Really !! How?

This is interesting. When a page is opened via a backlink provided it has the context of parent page via window.opener property.

That malicious website can do anything like

1. Accessing the cookies window.opener.document.cookie will get access to all the cookies
2. Manipulating the DOM
3. Replace it completely with a similar page and do Phishing !! window.opener.location = www.something-similar.com

The Options are endless.

Ugh !! How to fix it?

To prevent it from happening, when adding a backlink you should also add rel='noopener'

<a href="external-site.com" target="_blank" rel="noopener">Reference</a>

How did this work?

It removes the reference to the parent page. When the malicious website tries to access it, it returns 'null' and your website is saved from those prying eyes of hacker.

To know more about all the options for the 'rel' property

https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel

But there is a catch. util.promisify() changes this context to golbalThis. So if the actual method is using this somewhere in the code, it will fail/produce undesired results.

mailer.sendSMTP = function(message, cb){
        // some code 
        this.smtpTransport.sendMail(message , cb);
    };

The above code will work perfectly fine when it is invoked with a callback fn. But when its promised as follows then it will throw TypeError: Cannot read property 'sendMail' of undefined

 try {
        emailResult = await promisify(mailer.sendSMTP)(emailToSend);
    } catch (error) {
        throw `Error while sending email: ${error}`;
    }

it can be fixed either by binding the context again or changing the implementation.

Binding the context using call, apply or bind methods

try {
        emailResult = await promisify(mailer.sendSMTP).call(mailer,emailToSend);
    } catch (error) {
        throw `Error while sending email: ${error}`;
    }

Moving the context variable outside of the method.

var _that = mailer; 
mailer.sendSMTP = function(message, cb){
        // some code 
        _that.smtpTransport.sendMail(message , cb);
    };

References: https://github.com/nodejs/node/issues/13338

But there are so many hidden gems inside our terminal that will elevate our skills and productivity.

In this series part 1, I'm gonna share a few things that will work for all the commands.

Most of the times we work inside the terminal in following patterns

1. Same Command with different arguments
2. Different command with the same arguments

In this case, we either retype the whole thing or edit the previous command/input. With the following set of tricks, we can do better.

 ! is the operator which is going to help us in these scenarios

 ⇄  Tab Or ↵ enter can be used to replace these substitutions

Same Command with different arguments

!! Operator - Substitues Last Command

Many times we would have run to a scenario where admin rights are needed for command.

>cat system.log
You need administrator access

and then we add sudo and type the command again. But there is a better way, type sudo !! and enter ↵ or ⇄ Tab

> sudo !! 
sudo cat system.log

<log file content>

We can also add alias su='su !!' to our profile

!:0 - Previous Command without arguments

To repeat the previous command with a different argument then type !:0 and press ↵ enter that will insert the previous command without the arguments and modifiers

> cat abc.txt
> !:0 (↵ enter) 
> cat |

To run it with some parts then !:0-n will get till the nth part of it

> ssh -i dev-uswest.pem ubuntu@10.801.118.256
    0  1  2              3

> !:0-2  (↵ enter)

> ssh -i dev-uswest.pem |

!prefix - Repeat previous command that starts with a specific word/letter

To repeat the last command which starts with ssh then

>!ssh (↵ enter)
>ssh -i uswest.pem ubuntu@10.01.118.256

!?str? - Last command containing str

To repeat the command which contains sedhu from the history

> !?sedhu?
> ssh -i uswest.pem sedhu@10.801.118.256

Ctrl + R Reverse Search

Search through the commands history and Keep Pressing Ctrl + R and Ctrl+Shift+R to continue searching Forward and Backward.

~/.aws ⌚ 21:31:24
$ ssh -i uswest.pem ubuntu@100.81.112.257
bck-i-search: ssh_

!n - Nth command from the history

If we remember the command history position, we use !n and !-n to execute the command in position from Top and Bottom of the list.

If the history is as follows

> history
    1  git push -f --no-verify
    2  npmR test
    3  npmR start:dev
    4  npmR test

then results of such commands will be

> !1 // git push -f --no-verify
> !-1 // npmR test
> !-2 // npmR start:dev

Different command with the same arguments

!$ Substitutes last argument

Let's say we were viewing a config file with the cat command

/etc/vim ⌚ 15:12:23
$ cat vimrc 
<vimrc file content>

now we need to edit something in that, type vi !$ and ↵ Enter

/etc/vim ⌚ 15:12:31
$ vi !$ (↵ enter)

/etc/vim ⌚ 15:14:50
$ vi vimrc

Esc . (Dot) - Inserts last argument

Suppose we need to inspect before executing that command then type Esc . which will insert the argument

/etc/vim ⌚ 15:15:32
$ vi (Esc .) 
$ vi vimrc

Few more

• !^ - Substitutes first argument from the previous command
• !* - Sustitues all arguments from the previous command
• !n - Nth Argument from the previous command
• !-n - Nth Argument from the previous command (Backwards)
• ^val1^val2 - Last command replacing val1 for val2

> ssh -i uswest.pem ubuntu@10.81.118.219
> ^ubuntu^sedhu (↵ enter)
> ssh -i uswest.pem sedhu@10.81.118.219

So I had to extend it and build our own NPM library so that it could be consumed by our products.

One of the main functionality which was missing was prefix - mask conversion and validation. As usual, I googled it, Unfortunately I didn't find any solution. So I had to implement it on my own and i thought it could be useful for the community.

Prefix to Mask Conversion

Input will be a number between 0 and 128 and the function should return the corresponding IPv6 Mask.

Explanation:

1. Create a 128 bit binary string
2. Fill the first n (prefix) bits with 1s and the remaining with 0s.
3. Chunk them with a size of 16 bits each
4. Convert each chuck to a hex value and join them with (:)

For Example: 118 => 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:fc00'

const HEX = 16;
const BINARY = 2;
const MAX_PREFIX = 128;
const MIN_PREFIX = 0;

  static isValidPrefix(prefix: number): boolean {
    return Number.isInteger(prefix) && prefix >= MIN_PREFIX && prefix <= MAX_PREFIX;
  }
  /**
   *
   * @param prefix
   * @returns ipv6 netmask address
   *
   * Fill an array with 1s for given number of prefix bits
   * Fill the remaining bits with 0s
   * chunk it with 16 elements and covert each chunk to hex
   *
   */
  static getNetmaskForPrefix(prefix: number): string {
    if (!IPV6.isValidPrefix(prefix)) {
      throw new TypeError('Invalid IPv6 Prefix');
    }

    const prefixArr: number[] = new Array(prefix).fill(1);
    const chunkArr = Array.from({
      length: Math.ceil(prefixArr.length / HEX),
    }, (_v, i) => prefixArr.slice(i * HEX, i * HEX + HEX));

    // Converting from binary to hex
    let subnet = chunkArr.map((item) => {
      return parseInt(item.join('').padEnd(HEX, '0'), BINARY).toString(HEX);
    }).join(':');

    if (subnet.length < 35) {
      subnet = `${subnet}::`;
    }

    return new Address6(subnet).canonicalForm();
  }

Mask to Prefix Conversion

This is the opposite of the previous function. With a given mask it should return the corresponding prefix.

Ex: expect(IPV6.getPrefixForNetmask('ffff:ffff:ffff:ffff:ffff:fffc:0000:0000')).toBe(94);

Explanation:

1. Convert the mask to binary using ip-address.js function.
2. Count the number of 1's and return the count as result.

/**
   *
   * @param {string} netmask
   * @returns {number} prefix
   *
   * Returns exception if its an invalid mask
   */
  static getPrefixForNetmask(netmask: string): number {

      if (IPV6.isValidMask(netmask)) {
        const bits = new Address6(netmask).getBitsBase2(0, 128); // Note : Address6 from ip-address.js 
        const masked = bits.match(/1/g);

        return masked ? masked.length : 0;
      }

      throw new TypeError('Invalid IPv6 Mask');
  }

Valid Mask

But before attempting to get the prefix out of the mask, we also need to check whether it is a valid mask, else it might return the wrong results.

Ideally, a valid mask can have a maximum of one flip bit in it and starts with 1 with trailing 0s or starts with 0 and no trailing 1s.

Example: 1100 => valid // starts with 1 with trailing 0s 0000 => valid // with 0 and no trailing 1s 1010 => Invalid 0001 => Invalid

Explanation:

1. Convert the mask to binary string
2. Iterate through each bit to count the flips.
3. If the count is more than 1, then it's not a valid mask.

static isValidMask(mask: string): boolean {
    try {
      const bits = new Address6(mask).getBitsBase2(0, 128);
      let flips = 0;
      let oldBit = '1';

      for (let i = 0; i < bits.length; i++) {
        if (oldBit !== bits.charAt(i)) {
          flips += 1;
          oldBit = bits.charAt(i);
        }

        if (flips > 1) {
          return false;
        }
      }

      return flips <= 1;
    } catch (e) {
      return false;
    }
  }

Bonus : Random IPv6 Address Generator ( used for testing)

Sometimes I needed a random IPv6 address for testing.

Explanation:

1. Create 7 * 4 chars chunks of individual sets with Math.random() and valid characters from IPv6 char set [0-9][a-f].
2. Append them with the given prefix and join it with the colon(:).

/**
     *
     * @param ipPrefix defaults to 2000:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx
     * @returns a random IPv6 address with the given prefix
     */
    static generateRandomIPv6Address(ipPrefix = '2000'): string {
        const items = '0123456789abcdef'.split('');
        const randomised = [];

        for (let i = 0; i < 28; i++) {
            randomised[i] = items[Math.floor(Math.random() * items.length)];
        }

        const ipBlock: string[][] = [];

        for (let i = 0; i < randomised.length; i++) {
            const blockIndex = Math.floor(i / 4);

            if (!Array.isArray(ipBlock[blockIndex])) {
                ipBlock.push([]);
            }

            if (!(ipBlock[blockIndex].length === 0) || randomised[i] !== '0') {
                ipBlock[blockIndex].push(randomised[i]);
            }
        }

        const ip: string[][] = [];

        for (let i = 0; i < ipBlock.length; i++) {
            const ipIndex = Math.floor(i / 8);

            if (!Array.isArray(ip[ipIndex])) {
                ip.push([]);
            }

            ip[ipIndex].push(ipBlock[i].join(''));
        }

        let suffixIp = '';

        for (let i = 0; i < ip.length; i++) {
            suffixIp += ip[i].join(':');
        }

        return `${ipPrefix}:${suffixIp}`;
    }