FAQ - Frequently Asked Questions

電腦科學基礎：程式/進程/執行緒、記憶體管理與 GC、作業系統核心概念（排程、死鎖、IPC、虛擬記憶體）、網路協定、資料結構與演算法複雜度速查。

Read more →

Covers: 0. Data Engineer Interview Questions With Python · 1. Difference between file format and storage system ? · 2. Difference between Cluster and Node?

the high-level map of a data engineer interview and the last-minute checklist: for each topic, the handful of points you must be able to say out loud, and a link to the deep dive under de/ that holds the mechanism, the code and the…

Read more →

Covers: 1) What are the different design schemas in Data Modelling? Explain with the example? · 2) Which schema is better – star or snowflake? · 3) What is a fact, dimension table ?

Covers: What a behavioral round assesses · Why it matters · How it’s scored

Covers: 1. Docker 基礎 · 容器 vs 虛擬機 · Dockerfile 重要概念

Practical, language-neutral notes on how to find and fix performance problems, with Java-specific tips at the end.

the Python language itself: object model, mutability, functions and decorators, iterators, OOP, memory management, typing, packaging and the gotchas interviewers actually probe.

how Python actually runs concurrent work: the GIL, threads vs processes vs asyncio, the synchronisation primitives, and how to pick between them.

How a program is loaded, laid out in memory, and executed — the concepts that come up in systems and language-internals interviews.

the analytics knowledge a data engineering interview expects, and how to talk about analytics experience: what "analytics experience" questions probe, defining metrics precisely, ratio metrics and the averaging trap, the common product…

the AWS data stack as a data engineering interview probes it: how S3, the Glue Data Catalog, Glue ETL, EMR, Athena and RDS fit together, choosing between the compute services, Glue jobs and bookmarks, EMR cluster anatomy and EMR…

the service-selection round of a senior data engineering interview on AWS: which service owns which layer, and how to choose and defend one against its look-alikes — S3 vs EBS vs EFS, RDS vs DynamoDB vs Redshift, Kinesis Data Streams vs…

everything a data engineer interview asks about batch pipelines: the reference architecture, ETL vs ELT, lake vs warehouse vs lakehouse, ingestion patterns, idempotency and backfills, late data, MERGE, orchestration with Airflow, dbt, data…

logical data modeling for analytics and the interview questions it draws: modeling levels, ER and normalization, OLTP vs OLAP, Kimball dimensional modeling (grain, facts, dimensions, the bus matrix), fact and dimension patterns, slowly…

the performance questions a data engineering interview asks across engines rather than about one: a method for any "make it faster" question, optimising how data is stored, optimising how it is processed, avoiding out-of-memory failures in…

Apache Flink as a data engineering interview probes the engine itself: the runtime (JobManager, TaskManagers, slots, chaining), deployment, the DataStream and Table / SQL APIs, watermarks and timers in code, state backends, checkpoint and…

the Hadoop ecosystem as a data engineer interview probes it: HDFS internals, MapReduce end to end, YARN, Hive, HBase, ZooKeeper, the satellite tools, file formats, and how the stack moved to object storage and the lakehouse.

Kafka as a data engineer is interviewed on it: the storage log, replication, the KRaft controller, producer and consumer internals, delivery semantics and transactions, retention and compaction, sizing, Kafka Connect and CDC, Schema…

physical schema design for data systems: how to answer a schema-design question, OLTP tables, constraints and indexes, keys and ID generation, warehouse partitioning / clustering / distribution, data types, NULLs and conventions, soft…

everything about Apache Spark a data engineering interview probes: the execution model, the APIs and the optimiser, shuffles, partitioning, joins and skew, caching, memory and serialisation, PySpark specifics, writes, Structured Streaming…

reading query plans as a data engineering interview probes it: what EXPLAIN, EXPLAIN ANALYZE and BUFFERS report, how to walk the plan tree, scan nodes, estimated vs actual rows, the three join algorithms as they appear in a plan, sort and…

analytical SQL as a data engineering interview tests it: how a query is evaluated, joins and their algorithms, NULL semantics, aggregation, window functions, CTEs and recursion, the classic interview query patterns, MERGE and upserts…

a question bank for the SQL round of a senior data engineering interview: every question is asked the way an interviewer asks it, then answered — the sentence to say out loud, the SQL, each follow-up, the cost at scale and the production…

relational databases and the NoSQL families side by side: transactions and isolation, storage engines, replication and sharding, CAP and consistency models, one representative store per family, query-first modeling, NewSQL, and how to…

everything a data engineer interview asks about stream processing: when to stream, reference architectures, time semantics and watermarks, windows and late data, state and checkpoints, exactly-once end to end, stream joins, backpressure…

splitting a system's write model from its read model: what CQRS buys (simplicity, scalability, speed), what it costs (eventual consistency, projectors, duplication), and when it is worth it.

OOP fundamentals for Java interviews: the four pillars, interface vs abstract class, composition over inheritance, SOLID and the everyday principles, and the object contracts.

core language mechanics: primitives vs references, strings, initialisation order, Object's methods, nested classes, reflection, I/O and the date/time API.

the collections framework: the hierarchy, ArrayList vs LinkedList, HashMap and ConcurrentHashMap internals, and the Big-O of every operation.

the GoF patterns that actually come up in Java interviews and code review, each with a minimal snippet and the place the JDK or Spring already uses it.

the Throwable hierarchy, checked vs unchecked, try/catch/finally mechanics, try-with-resources, and the design rules for exceptions in application code.

the Java 8 functional toolkit: lambdas, method references, the built-in functional interfaces, the Stream API, collectors, and Optional.

type parameters, type erasure and everything that follows from it: bounded types, wildcards and PECS, generic methods, and why generic arrays are illegal.

what changed after Java 8: var, records, sealed types, pattern matching, text blocks, the module system, and virtual threads.

writing correct concurrent Java: thread lifecycle, synchronized and ReentrantLock, CAS and atomics, thread pools, CompletableFuture, ThreadLocal and the concurrent collections.

the container (IoC/DI, bean lifecycle and scopes), AOP, @Transactional semantics, Spring MVC's request path, Spring Boot auto-configuration, and testing.

test-driven development in Java: the red-green-refactor cycle, JUnit 5, the AAA structure, mocking, and what is worth testing.

the rules for when one thread's write becomes visible to another: atomicity, visibility, ordering, happens-before, and what volatile, synchronized and final each guarantee.

how the JVM stores, executes and reclaims: runtime memory areas, object creation, garbage collection and collectors, reference strengths, class loading, JIT, and the flags/tools used to diagnose a sick JVM.

designing an HTTP API: resource modelling, status codes, idempotency, pagination, versioning, error shape, auth, and how REST compares with GraphQL and gRPC.

Covers: 1. Session（傳統方式） · 運作流程 · Session 儲存位置

Covers: High Traffic Patterns · 1. Idempotent Request Processing · Initial Design (V0 - Interview Whiteboard)

Covers: Thread Pool & Executor Framework · 1. ThreadPoolExecutor Deep Dive · 2. Rejection Policies Comparison

Covers: Advanced Concurrency · 1. Fork/Join Framework · 2. Lock-Free Programming

Covers: 設定隔離級別 · Session 級別（當前連線） · 全域級別（影響所有新連線）

常見 4xx / 5xx 錯誤碼各代表什麼、常見成因、client 端與 server 端該怎麼處理與排查（含哪些可以重試）；不談設計 API 時該回哪個狀態碼與錯誤格式。

Covers: 1. 什麼是冪等性 · 2. 為什麼 LLM 工具特別需要冪等性 · 3. 基礎實作：Check-Act-Record

Covers: Problem · Strategies · 1. Atomic DECR (Redis)

Covers: 1. HTTP Keep-Alive (Persistent Connection) · 2. HTTP Pipelining · 3. Long Polling

設計「接收端」的 webhook 整合：簽章驗證、去重、快速 ACK、非同步處理、重試與 DLQ、亂序事件與可觀測性；不談一般 HTTP API 的設計，也不談長連線推播。

Covers: 1. 資料庫相關 · 1.1 資料庫隔離級別 (Transaction Isolation Levels) · ACID 特性

Covers: 1) DB type · 2) DB properties · 3) DB design

Read more →

Covers: 1) array_to_string : array to str · 2) cast : type transform · 2’) to_char : type transform

Covers: Dirty read, Lost of modify, Unrepeatable read, Phantom read · MySQL index · MySQL explain (execution plan)

Covers: 1. Given 2 table a (3k rows), b (4k rows), what’s the · 2. Difference between full outer Join And Union in SQL ? · 3. Find the orders in Quater ? (no hard-code)

Covers: Redis data structure · Redis部署模式 · Why redis is fast?

Redis is a powerful, open-source in-memory data structure store that is widely used in modern backend systems. It is often utilized for caching, session storage, real-time analytics, and messaging systems, among other use cases.

Covers: Sorted set (ZSET) · 🧠 Redis Sorted Set Primer · ✅ Basic Leaderboard Operations with Redis

Covers: 1) Explain kafka architecture ? · 1’) Kafka message structure ? · 1’') How kafka find .log file via index ?

a reading list for Hadoop: HDFS architecture, a classic interview question list, HA setup and node commissioning. The answers themselves are in the deep dive.

a reading list on expressing SQL queries as MapReduce jobs. The MapReduce model itself is answered elsewhere.

the classic Spark / Hadoop interview questions, each answered in the few sentences you would say out loud, plus the troubleshooting walk-throughs, pipeline-design prompts and study plan that turn them into a drill.

Older notes, kept for their diagrams. For interview preparation start with the deep dive, ../de/flink.md.

Covers: Simple stream from scratch · Merge multiple input stream into one