โก Interactive Practice Engine: 50 Rapid-Fire Architecture Drills
Welcome to the Staff Platform Architect & Principal SRE Rapid-Fire Drill Engine.
Use this diagnostic simulator to test your fast-recall instincts under interview conditions. Each drill presents a real-world production symptom, failure signature, or architectural trade-off commonly tested in Indian tech interviews.
๐งญ Step-by-Step Rapid-Fire Drill Playbook
- Step 1 (Set a 60-Second Timer Per Scenario): Read the failure signature and state your root-cause hypothesis and first CLI triage command within 60 seconds.
- Step 2 (Expand the Solution): Click the expandable solution block and score yourself strictly across all 3 points (Root Cause, Triage Command, Permanent Fix).
- Step 3 (Review Linked Playbooks): For any score below 3/3, jump to the corresponding Troubleshooting or Round module and practice the step-by-step commands.
๐ฏ Scoring & Hiring Readiness Rubric
| Correct Answers | Evaluated Level | Target Companies & Roles |
|---|---|---|
| 45 โ 50 | Principal Architect / Distinguished SRE (L7+) | Google Cloud, Uber IDC, Atlassian, Stripe, PhonePe Staff Architect |
| 38 โ 44 | Staff Platform Engineer / Lead SRE (L6) | Razorpay, CRED, Swiggy, Amazon AWS, Microsoft IDC, Goldman Sachs |
| 28 โ 37 | Senior Platform Engineer (L5) | High-growth scaleups, Enterprise MNCs, Walmart Global Tech |
| < 28 | Developing / Foundations Review Needed | Review Round 1 through Round 6 modules |
๐ง Section 1: Linux & Low-Level Networking (Drills 1โ10)
1. chmod 755 vs chmod 644 on a Directory
Why does executing chmod 644 on a directory prevent users from running cd into that directory even though Read (r) permission is set?
๐ก ๐ Solution & Rationale
Execute (x) permission on a directory is required to traverse (enter/cd) into it or look up inodes within it. Read (r) only allows reading the directory contents (ls). Without x, any attempt to cd fails with Permission denied.
2. Linux Load Average Mystery
Why can Linux Load Average (reported by uptime) be 15.0 on a 4-core machine while CPU utilization is only 10%?
๐ก ๐ Solution & Rationale
Linux Load Average includes processes in both TASK_RUNNING and TASK_UNINTERRUPTIBLE (D state). A high load average with low CPU utilization indicates processes blocked on disk I/O, NFS mounts, or uninterruptible kernel locks.
3. Ephemeral Port Range
What is the standard Linux ephemeral port range and how many concurrent connections to a single destination IP:Port can a host support?
๐ก ๐ Solution & Rationale
Default range is 32768โ60999 (approx. 28,231 ports). To expand, set net.ipv4.ip_local_port_range = 1024 65535 (approx. 64,511 ports). One host can maintain up to 64,511 simultaneous outbound connections to a single remote IP:Port tuple.
4. TCP CLOSE_WAIT vs TIME_WAIT
If you observe 10,000 sockets stuck in CLOSE_WAIT, is this an application bug or an infrastructure issue?
๐ก ๐ Solution & Rationale
It is an application bug. CLOSE_WAIT means the remote client sent a FIN and the kernel acknowledged it, but the local application process has failed to call close() on the socket file descriptor (socket leak).
5. tcp_tw_reuse vs tcp_tw_recycle
Why was tcp_tw_recycle removed from the Linux kernel while tcp_tw_reuse remains safe?
๐ก ๐ Solution & Rationale
tcp_tw_recycle dropped packets from different clients sharing a common NAT Gateway due to desynchronized TCP timestamps. tcp_tw_reuse only applies to outbound client connections where timestamps are locally generated and monotonic.
6. CFS CPU Quota Period
What is the default Linux CFS period in Kubernetes and what file tracks throttling?
๐ก ๐ Solution & Rationale
Default period is 100ms (cpu.cfs_period_us = 100000). Throttling metrics are tracked in /sys/fs/cgroup/cpu/cpu.stat via nr_throttled and throttled_time.
7. DNS ndots:5 Impact
If an application queries service.internal, how many DNS queries does glibc perform before querying the root domain?
๐ก ๐ Solution & Rationale
service.internal has 1 dot. Since 1 < 5, glibc sequentially queries all search paths in /etc/resolv.conf (e.g. namespace.svc.cluster.local, svc.cluster.local, cluster.local) before querying service.internal. directly.
8. PMTUD Blackhole
What ICMP message type and code is required for Path MTU Discovery to function?
๐ก ๐ Solution & Rationale
ICMP Type 3, Code 4 (Destination Unreachable, Fragmentation Needed and DF Set). Blocking this ICMP packet in firewalls causes PMTUD blackholes.
9. Container PID 1 Signal Handling
Why does a Node.js process running as PID 1 in a container fail to shut down when Kubernetes sends SIGTERM?
๐ก ๐ Solution & Rationale
Linux disables default signal handlers for PID 1. If the application has not explicitly registered a handler for SIGTERM, the kernel discards the signal. Use tini or dumb-init.
10. conntrack Table Saturation
What sysctl controls maximum conntrack table size and what happens when it fills up?
๐ก ๐ Solution & Rationale
Controlled by net.netfilter.nf_conntrack_max. When saturated, the kernel drops incoming packets for new connections, incrementing net.netfilter.nf_conntrack_insert_failed.
โธ๏ธ Section 2: Kubernetes Internals & Scheduling (Drills 11โ20)
11. etcd Quorum Math
How many nodes can fail in a 5-node etcd cluster without causing service disruption? What about a 6-node cluster?
๐ก ๐ Solution & Rationale
- 5 nodes: Quorum is โ 5/2 โ + 1 = 3. It tolerates 2 node failures.
- 6 nodes: Quorum is โ 6/2 โ + 1 = 4. It still tolerates only 2 node failures while increasing network replication latency. Never use even-numbered clusters.
12. Pod QoS Classes
Name the 3 Kubernetes QoS classes and their oom_score_adj values.
๐ก ๐ Solution & Rationale
- Guaranteed:
requests == limitsfor all containers.oom_score_adj = -997. - Burstable:
requests < limits.oom_score_adjis dynamically calculated between2and999. - BestEffort: No requests or limits.
oom_score_adj = 1000(killed first).
13. EBS CSI Multi-AZ Storage Trap
Why does a StatefulSet pod using an EBS volume fail to start when moved to a new worker node?
๐ก ๐ Solution & Rationale
EBS volumes are physically constrained to a single Availability Zone. If the replacement node is launched in a different AZ, the volume cannot attach. Fix by setting volumeBindingMode: WaitForFirstConsumer on the StorageClass.
14. Mutating vs Validating Webhook Order
In what order does kube-apiserver execute Mutating and Validating admission webhooks?
๐ก ๐ Solution & Rationale
Mutating webhooks execute FIRST (sequentially) so they can modify or inject fields. Validating webhooks execute LAST (in parallel) to verify the final mutated state.
15. PodDisruptionBudget (PDB) Node Drain Freeze
What happens when kubectl drain is run on a node hosting a pod protected by a PDB with minAvailable: 1 and 1 total replica?
๐ก ๐ Solution & Rationale
The eviction API rejects the pod eviction to respect minAvailable: 1. kubectl drain blocks indefinitely until the PDB is adjusted or another replica is launched.
16. Informer Cache & Level-Triggered Logic
Why does Kubernetes use Informers with Level-Triggered reconciliation rather than Edge-Triggered event queues?
๐ก ๐ Solution & Rationale
Edge-triggered systems react to state transitions and miss updates if an event is dropped. Level-triggered controllers reconcile current state against desired state, ensuring automatic convergence even after crashes or missed network messages.
17. Karpenter vs Cluster Autoscaler
Why is Karpenter faster at provisioning nodes than Cluster Autoscaler?
๐ก ๐ Solution & Rationale
Cluster Autoscaler scales AWS Auto Scaling Groups (ASGs), waiting for ASG cooldowns and Launch Template evaluations. Karpenter bypasses ASGs and directly invokes the AWS EC2 Fleet API, provisioning custom bin-packed instances in <45s.
18. CNI: AWS VPC CNI vs Cilium
What is the primary difference in how AWS VPC CNI and Cilium allocate pod IP addresses?
๐ก ๐ Solution & Rationale
- AWS VPC CNI assigns native AWS VPC secondary private IPs to ENIs attached to the EC2 host.
- Cilium uses an overlay or native routing IPAM, utilizing eBPF to route packets directly in the kernel without kube-proxy iptables rules.
19. Kubernetes Finalizers
What happens when a resource with a finalizer is deleted using kubectl delete?
๐ก ๐ Solution & Rationale
The API Server sets metadata.deletionTimestamp but does not delete the object from etcd. It remains in Terminating state until the managing controller completes cleanup and removes its string from metadata.finalizers.
20. HPA Metrics
Why should you never configure HPA to autoscale on memory utilization for Java workloads?
๐ก ๐ Solution & Rationale
JVM memory management retains allocated heap memory from the OS even after garbage collection. Memory usage rarely drops, causing HPA to scale out to max replicas and never scale down. Autoscale Java on CPU or request rate.
โ๏ธ Section 3: IaC, Multi-Cloud & Security (Drills 21โ30)
21. Terraform State Serial Number
What is the purpose of the serial field inside terraform.tfstate?
๐ก ๐ Solution & Rationale
serial is an incrementing integer. Every time Terraform writes a state modification, it increments serial. It acts as an optimistic concurrency lock: if the remote state serial is higher than the runnerโs local state, Terraform aborts to prevent overwrite collisions.
22. Terraform moved Block
What problem do Terraform moved blocks solve when refactoring modules?
๐ก ๐ Solution & Rationale
They instruct Terraform that a resource address changed (e.g. aws_s3_bucket.a to module.storage.aws_s3_bucket.b) without destroying and re-creating the physical cloud resource.
23. TGW Appliance Mode
What problem occurs when stateful firewalls are placed behind an AWS Transit Gateway without Appliance Mode?
๐ก ๐ Solution & Rationale
Asymmetric routing. Outbound packets flow through the firewall in AZ-a, but return packets hash to the firewall in AZ-b. Because Firewall B has no record of the initial TCP SYN, it drops the return packet.
24. EKS IRSA Projected Token
Where does kubelet mount the OIDC token used by AWS SDKs for IAM Roles for Service Accounts?
๐ก ๐ Solution & Rationale
At /var/run/secrets/eks.amazonaws.com/serviceaccount/token. The SDK passes this JWT to STS via AssumeRoleWithWebIdentity.
25. Vault Raft Storage Quorum
How does HashiCorp Vault with Integrated Storage (Raft) handle an isolated AZ in a 3-node cluster?
๐ก ๐ Solution & Rationale
The two surviving nodes in the non-isolated AZs retain quorum (2/3) and elect a new Raft leader within 300ms. The isolated node enters a follower state and rejects write requests.
26. AWS Gateway VPC Endpoints
Which two AWS services support free Gateway VPC Endpoints rather than billable Interface Endpoints?
๐ก ๐ Solution & Rationale
Amazon S3 and Amazon DynamoDB. They modify VPC route tables directly and do not incur hourly ENI or data processing fees.
27. Crossplane XRD vs Composition
What is the difference between a CompositeResourceDefinition (XRD) and a Composition in Crossplane?
๐ก ๐ Solution & Rationale
- XRD: Defines the custom API schema and type definition (the OpenAPI specification).
- Composition: Defines the implementation template that maps the XRD to real cloud managed resources (e.g., creating an RDS instance, S3 bucket, and IAM role).
28. OPA Gatekeeper vs Kyverno Language
What language does OPA Gatekeeper use for policy definitions, and what does Kyverno use?
๐ก ๐ Solution & Rationale
OPA Gatekeeper uses Rego (a declarative query language based on Datalog). Kyverno uses Kubernetes-native YAML.
29. Cosign Keyless Image Signing
What three components make up the Sigstore keyless signing ecosystem?
๐ก ๐ Solution & Rationale
- Fulcio: Ephemeral Certificate Authority issuing x509 certs based on OIDC tokens.
- Rekor: Immutable transparency log for signature proofs.
- Cosign: Client CLI for signing and verifying OCI container artifacts.
30. Terragrunt mock_outputs
Why are mock_outputs required in Terragrunt during cold-boot infrastructure provisioning?
๐ก ๐ Solution & Rationale
When provisioning account B that depends on outputs from account A before account A has run terragrunt apply, Terragrunt fails validation. mock_outputs injects temporary dummy values to allow terragrunt validate and plan to succeed.
๐จ Section 4: SRE, Observability & War Room (Drills 31โ40)
31. High Cardinality Definition
Define metric cardinality in Prometheus and provide an example of a dangerous label.
๐ก ๐ Solution & Rationale
Cardinality is the total number of unique time series produced by multiplying label value sets. Adding user_id, order_id, or client_ip creates millions of unique series, exhausting TSDB memory.
32. Multi-Window Multi-Burn-Rate Alerting
Why is a 14.4x burn rate over 1 hour paired with a 5-minute short window in Google SRE alerting?
๐ก ๐ Solution & Rationale
The 1-hour window detects serious 2% budget consumption over 30 days. The 5-minute short window ensures the alert automatically resolves if the outage was a temporary spike that already ended.
33. OpenTelemetry Head vs Tail Sampling
Why is Tail-Based Sampling superior to Head-Based Sampling for microservices tracing?
๐ก ๐ Solution & Rationale
- Head sampling decides whether to sample at request entry, dropping 99% of traces before knowing if they will result in an error.
- Tail sampling waits until the trace finishes, ensuring 100% of errors and high-latency traces are retained while sampling successful 200 OK traces at 1%.
34. Thundering Herd Problem
What is a thundering herd (dogpile effect) in caching systems?
๐ก ๐ Solution & Rationale
When a highly popular cached key expires, thousands of concurrent requests encounter a cache miss simultaneously and hammer the backend database at the exact same millisecond.
35. Probabilistic Early Invalidation (XFetch)
How does the XFetch algorithm prevent cache stampedes?
๐ก ๐ Solution & Rationale
It computes a probability of refreshing the cache before the key expires based on the computation time ฮด and remaining TTL. As TTL decreases, background requests refresh the key before it expires for users.
36. Database Connection Pool Starvation
Why does adding more Kubernetes pod replicas worsen a database outage during high latency?
๐ก ๐ Solution & Rationale
Each new pod replica initializes its own connection pool (e.g., 50 connections). 200 new pods attempt 10,000 new database connections, exceeding database max_connections and crashing the database engine.
37. Circuit Breaker States
Name the 3 states of a standard distributed circuit breaker.
๐ก ๐ Solution & Rationale
- Closed: Normal operation; requests pass through.
- Open: Failure threshold breached; requests fail fast immediately without calling downstream.
- Half-Open: Tests downstream recovery by allowing a small percentage of trial requests.
38. Distributed Tracing Context Propagation
What standard W3C HTTP header propagates distributed trace context across microservices?
๐ก ๐ Solution & Rationale
traceparent (format: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01).
39. Blameless Postmortem Principle
What is the core premise of a blameless postmortem regarding human error?
๐ก ๐ Solution & Rationale
Human error is a symptom of systemic failure, not the root cause. Focus on systemic vulnerabilities, missing automated guardrails, and psychological safety rather than individual punishment.
40. PromQL Rate vs Irate
What is the difference between PromQL rate() and irate()?
๐ก ๐ Solution & Rationale
rate()calculates the per-second average rate over the entire specified time window (smooth, ideal for alerts).irate()calculates the rate based on the last two data points in the range (volatile, ideal for zooming into fast micro-spikes on dashboards).
๐๏ธ Section 5: System Design, FinOps & Leadership (Drills 41โ50)
41. Spot Fleet Interruption Warning Window
How much advance warning does AWS EC2 provide before terminating a Spot instance?
๐ก ๐ Solution & Rationale
Exactly 2 minutes via EC2 Spot Instance Interruption Notices published to metadata and Amazon EventBridge / SQS.
42. AWS Graviton Price/Performance Advantage
What are the financial and performance benefits of migrating from x86 Intel to AWS Graviton processors?
๐ก ๐ Solution & Rationale
Graviton (ARM64) instances offer an immediate 20% direct price reduction and up to 20% higher performance per watt, yielding a combined ~40% price-performance gain.
43. EBS gp2 vs gp3 Architecture
Why is EBS gp3 superior to gp2 for cost optimization?
๐ก ๐ Solution & Rationale
gp3 is 20% cheaper per GB and decouples IOPS and throughput from volume size. With gp2, achieving 3,000 IOPS required provisioning a wasteful 1TB volume; gp3 provides 3,000 IOPS free on any volume size.
44. CAP Theorem in Multi-Region Architectures
Why is an Active-Active Multi-Region architecture with RPO=0 impossible across distant regions?
๐ก ๐ Solution & Rationale
Speed-of-light optical latency across 700+ km adds 15โ25ms to every synchronous write. Under the CAP Theorem, you cannot achieve both continuous Availability and immediate Consistency during cross-region network partitions.
45. Cell-Based Architecture
How does cell-based architecture eliminate cross-region write lock contention?
๐ก ๐ Solution & Rationale
Users are deterministically partitioned into isolated independent cells (e.g., by customer ID or geography). Each cell owns all primary data for its partitioned users, eliminating cross-region distributed locks.
46. Backstage Software Catalog Entity
What manifest file defines an entity inside the Spotify Backstage software catalog?
๐ก ๐ Solution & Rationale
catalog-info.yaml (defining apiVersion: backstage.io/v1alpha1, kind: Component, type: service, etc.).
47. vLLM PagedAttention Advantage
What problem does vLLM PagedAttention solve compared to standard HuggingFace Transformers?
๐ก ๐ Solution & Rationale
Standard transformers allocate contiguous GPU memory for the Key-Value (KV) cache, resulting in 60โ80% memory fragmentation. PagedAttention divides the KV cache into non-contiguous virtual memory blocks, unlocking 4x higher throughput.
48. Tensor Parallelism (TP) vs Pipeline Parallelism (PP)
When serving a 70B parameter model across 4 GPUs, do you use Tensor Parallelism or Pipeline Parallelism?
๐ก ๐ Solution & Rationale
Tensor Parallelism (TP=4) across GPUs on the same physical node connected via high-speed NVLink. It splits matrix multiplication within each layer, whereas Pipeline Parallelism splits layers across nodes and adds latency.
49. Architecture Decision Record (ADR)
What 5 sections comprise a standard ADR?
๐ก ๐ Solution & Rationale
- Title & Status (Proposed, Accepted, Superceded).
- Context (The problem and business drivers).
- Decision (The chosen architectural change).
- Consequences (Positive and negative trade-offs).
- Compliance / Verification (How adherence is monitored).
50. Error Budget Policy Action
What operational action occurs when a team burns 100% of its quarterly error budget?
๐ก ๐ Solution & Rationale
Feature freezes. Deployment of non-critical product features is halted. 100% of engineering bandwidth is redirected toward reliability hardening, automated test suites, and infrastructure debt until the error budget recovers.