Go to file
Tom Marshall 495d191d16 checkpoint: Mostly working
This seems to work except for I/O timing.  Reads are sync and
writes are async, so this sequence fails:
  - Object x flushes
  - Object x is reused as y
  - Another object is taken as x
  - New object x reads
  -> Stale data is read

Two potential solutions from here:

1. Implement async reads.

2. Hold ref to object until write completes.

(1) is complicated, but more correct.  Writes may stay in buffers for
quite some time (typically 5 seconds), during which time the dm-compress
object cannot be released.
2019-10-21 19:39:27 -07:00
cbd Fix a bunch of things 2019-10-10 13:27:37 -07:00
dm-compress checkpoint: Mostly working 2019-10-21 19:39:27 -07:00
include checkpoint: Mostly working 2019-10-21 19:39:27 -07:00
libcbd checkpoint: Mostly working 2019-10-21 19:39:27 -07:00
.gitignore Add a .gitignore 2019-10-09 11:35:16 -07:00
Makefile checkpoint: Mostly working 2019-10-21 19:39:27 -07:00
README Initial working version 2019-10-09 11:04:40 -07:00
TODO checkpoint: Mostly working 2019-10-21 19:39:27 -07:00

README

This is a device mapper block compression driver.

Note:
  - Sectors are always 512 bytes (kernel constant).
  - Physical blocks are always 4kb (internal constant).
  - Logical blocks are variable 4kb .. 1gb.
  - All integers are little endian.

Block device layout:
  - byte[4096] header
    - byte[4]               magic
    - u16                   version_major
    - u16                   version_minor
    - u64                   nr_physical_sectors
    - u64                   nr_logical_sectors [3 * physical_sectors]
    - u16                   physical_blocks_per_logical_block_shift (0 .. 18) [4 = 64kb]
                              (logical blocks may not exceed 1gb due to LZ4)
    - byte[]                reserved
  - byte[] zone ...         vector of zone

Number of logical blocks per zone:
  nr_lblk = 1 << header.logical_blocks_per_group_shift

Device size limits:
  - Min: header plus one zone (header, phys_alloc, log_alloc, data)
    Arbitrarily limit to 1mb (?)
  - 2^48 = 256t physical blocks = 1eb

There are six different lblk_alloc sizes:
  Struct name           Size    lblk_size       device_pblk_size
  ===========           ====    =========       ================
  lblk_alloc_16_16      2+2*8   <= 64kb         <= 64k
  lblk_alloc_16_32      2+4*8   <= 64kb         <= 4g
  lblk_alloc_16_48      2+6*8   <= 64kb         <= 256t
  lblk_alloc_32_16      4+2*8   > 64kb          <= 64k
  lblk_alloc_32_32      4+4*8   > 64kb          <= 4g
  lblk_alloc_32_48      4+6*8   > 64kb          <= 256t


Zone layout:
  - byte[4k]                Physical block alloc bitmap
  - lblk_alloc_x_y[]        Logical block allocation
  - data[4k*4k*8]           Data

  Data size (physical): 4kb*(4k*8) = 128mb
  Data size (logical) : 128mb*3 = 384mb

Assume <=4g physical blocks, 64kb logical blocks, 3x compression:
  - Zone holds 3*128mb = 384mb logical data
  - 384mb/64kb = 6k logical blocks
  - 6k * (34b) = 204kb logical block alloc data
  - 204kb / 4kb = 51 physical blocks for logical block alloc data
  - Overhead: (1+51)/32k = 0.016%