中分析并验证了MySQL进行在线创建索引时，不会因为执行时间过长或业务压力较大，在回放增量DML时加锁时间过久而对业务造成严重影响，本文从MySQL 8.0.19源码出发，分析MySQL是如何实现的。同时也确认是否在回放DML时会报duplicate key。

核心处理流程和对象

增量DML处理流程主要在http://row0log.cc中。

/** @file row/row0log.cc
 Modification log for online index creation and online table rebuild

 Created 2011-05-26 Marko Makela

通过阅读本文件代码可以发现，创建二级索引的增量DML记录和回放流程跟其他类型DDL是分开的。因为二级索引创建不需要重建表。

创建二级索引的处理流程

缓存增量DML操作

row_log_online_op用于处理创建二级索引的增量DML：

/** Logs an operation to a secondary index that is (or was) being created. */
void row_log_online_op(
    dict_index_t *index,   /*!< in/out: index, S or X latched */
    const dtuple_t *tuple, /*!< in: index tuple */
    trx_id_t trx_id)       /*!< in: transaction ID for insert,
                           or  for delete */
{

该函数直接调用os_file_write_int_fd将日志写入到临时文件中： 

err = os_file_write_int_fd(request, "(modification log)", log->fd,
                               log->tail.block, byte_offset, srv_sort_buf_size);

该场景下，写入临时文件的内容为正在创建的二级索引记录，无需写入聚集/主键索引记录。这样可以大大减少临时文件的数据写入量，二级索引记录构造函数如下： 

rec_convert_dtuple_to_temp(b + extra_size, index, tuple->fields,
                             tuple->n_fields, NULL);

回放增量DML操作

创建二级索引的增量DML回放入口为row_log_apply：

/** Apply the row log to the index upon completing index creation.
@param[in]  trx transaction (for checking if the operation was
interrupted)
@param[in,out]  index   secondary index
@param[in,out]  table   MySQL table (for reporting duplicates)
@param[in,out]  stage   performance schema accounting object, used by
ALTER TABLE. stage->begin_phase_log_index() will be called initially and then
stage->inc() will be called for each block of log that is applied.
@return DB_SUCCESS, or error code on failure */
dberr_t row_log_apply(const trx_t *trx, dict_index_t *index,
                      struct TABLE *table, ut_stage_alter_t *stage) {

...
  rw_lock_x_lock(dict_index_get_lock(index));

  if (!index->table->is_corrupted()) {
    error = row_log_apply_ops(trx, index, &dup, stage);
  } else {
    error = DB_SUCCESS;
  }
  ...
   } else {
    ut_ad(dup.n_dup == );
    dict_index_set_online_status(index, ONLINE_INDEX_COMPLETE);
  }

  log = index->online_log;
  index->online_log = NULL;
  rw_lock_x_unlock(dict_index_get_lock(index));

从这段代码可以发现调用row_log_apply_ops实际执行增量DLM回放前加了对应二级索引的互斥锁。回放完成，将索引状态设置为ONLINE_INDEX_COMPLETE，后进行解锁。

我们在测试过程已经发现，增量回放过程是不会加长时间锁的，这跟代码实现似乎有冲突。我们在后面分析row_log_apply_ops小结揭晓。

表重建场景的处理流程

该场景不是本文分析重点，在此简单说明下。

对于需重建表的DDL场景，DML操作处理函数分别为row_log_table_apply_insert、row_log_table_apply_update和row_log_table_apply_delete，在函数的注释上进一步注明了用于回放对应DML操作的函数：

/** Logs an insert to a table that is being rebuilt.
 This will be merged in row_log_table_apply_insert(). */
void row_log_table_insert(
    const rec_t *rec,       /*!< in: clustered index leaf page record,
                            page X-latched */
    const dtuple_t *ventry, /*!< in: dtuple holding virtual column info */
    dict_index_t *index,    /*!< in/out: clustered index, S-latched
                            or X-latched */
    const ulint *offsets)   /*!< in: rec_get_offsets(rec,index) */
{
  row_log_table_low(rec, ventry, NULL, index, offsets, true, NULL);
}

/** Logs an update to a table that is being rebuilt.
 This will be merged in row_log_table_apply_update(). */
void row_log_table_update(
    const rec_t *rec,          /*!< in: clustered index leaf page record,
                               page X-latched */
    dict_index_t *index,       /*!< in/out: clustered index, S-latched
                               or X-latched */
    const ulint *offsets,      /*!< in: rec_get_offsets(rec,index) */
    const dtuple_t *old_pk,    /*!< in: row_log_table_get_pk()
                               before the update */
    const dtuple_t *new_v_row, /*!< in: dtuple contains the new virtual
                             columns */
    const dtuple_t *old_v_row) /*!< in: dtuple contains the old virtual
                             columns */
{
  row_log_table_low(rec, new_v_row, old_v_row, index, offsets, false, old_pk);
}

/** Logs a delete operation to a table that is being rebuilt.
 This will be merged in row_log_table_apply_delete(). */
void row_log_table_delete(
    trx_t *trx,             /*!< in: current transaction */
    const rec_t *rec,       /*!< in: clustered index leaf page record,
                            page X-latched */
    const dtuple_t *ventry, /*!< in: dtuple holding virtual column info */
    dict_index_t *index,    /*!< in/out: clustered index, S-latched
                            or X-latched */
    const ulint *offsets,   /*!< in: rec_get_offsets(rec,index) */
    const byte *sys)        /*!< in: DB_TRX_ID,DB_ROLL_PTR that should
                            be logged, or NULL to use those in rec */

上述3个函数均是调用row_log_table_close执行增量DML日志格式化和写入操作： 

/** Stops logging an operation to a table that is being rebuilt. */
static void row_log_table_close_func(
    row_log_t *log, /*!< in/out: online rebuild log */
#ifdef UNIV_DEBUG
    const byte *b, /*!< in: end of log record */
#endif             /* UNIV_DEBUG */
    ulint size,    /*!< in: size of log record */
    ulint avail)   /*!< in: available size for log record */
{

该函数再后调用os_file_write_int_fd 

err = os_file_write_int_fd(request, "(modification log)", log->fd,
                               log->tail.block, byte_offset, srv_sort_buf_size);

总的来说，创建二级索引和重建表都需要处理增量DML，但处理方式不一样，相对来说，创建二级索引场景更加简单，因为只需要处理新增的二级索引记录即可。

增量DML记录和回放核心对象

row_log_t对象

不管是创建二级索引还是进行表重建，处理增量DML的核心对象都是row_log_t，该对象具体内容如下所示：

/** @brief Buffer for logging modifications during online index creation

All modifications to an index that is being created will be logged by
row_log_online_op() to this buffer.

All modifications to a table that is being rebuilt will be logged by
row_log_table_delete(), row_log_table_update(), row_log_table_insert()
to this buffer.

When head.blocks == tail.blocks, the reader will access tail.block
directly. When also head.bytes == tail.bytes, both counts will be
reset to 0 and the file will be truncated. */
struct row_log_t {
  int fd;              /*!< file descriptor */
  ib_mutex_t mutex;    /*!< mutex protecting error,
                       max_trx and tail */
  page_no_map *blobs;  /*!< map of page numbers of off-page columns
                       that have been freed during table-rebuilding
                       ALTER TABLE (row_log_table_*); protected by
                       index->lock X-latch only */
  dict_table_t *table; /*!< table that is being rebuilt,
                       or NULL when this is a secondary
                       index that is being created online */
  bool same_pk;        /*!< whether the definition of the PRIMARY KEY
                       has remained the same */
  const dtuple_t *add_cols;
  /*!< default values of added columns, or NULL */
  const ulint *col_map; /*!< mapping of old column numbers to
                        new ones, or NULL if !table */
  dberr_t error;        /*!< error that occurred during online
                        table rebuild */
  trx_id_t max_trx;     /*!< biggest observed trx_id in
                        row_log_online_op();
                        protected by mutex and index->lock S-latch,
                        or by index->lock X-latch only */
  row_log_buf_t tail;   /*!< writer context;
                        protected by mutex and index->lock S-latch,
                        or by index->lock X-latch only */
  row_log_buf_t head;   /*!< reader context; protected by MDL only;
                        modifiable by row_log_apply_ops() */
  ulint n_old_col;
  /*!< number of non-virtual column in
  old table */
  ulint n_old_vcol;
  /*!< number of virtual column in old table */
  const char *path; /*!< where to create temporary file during
                    log operation */
};