The one thing to understand first
An ordinary CREATE INDEX takes a SHARE lock on the table for its entire duration. That lock permits reads but blocks all writes — every INSERT, UPDATE, and DELETE waits until the index finishes building, which on a large table can be many minutes. CREATE INDEX CONCURRENTLY (CIC) exists to build the index without blocking writes.
CONCURRENTLY trades wall-clock speed for availability: it scans the table twice and waits for old transactions to finish, but never blocks your writes — and if it fails it leaves an INVALID index you must clean up. Those two facts (the wait, and the INVALID-index aftermath) are where every CIC surprise comes from.
How CONCURRENTLY works
CIC trades speed for availability by doing two passes over the table with only a weak SHARE UPDATE EXCLUSIVE lock (which allows concurrent DML), defined in src/backend/commands/indexcmds.c: