Commit Graph

29 Commits

Author SHA1 Message Date
Tom Marshall 90a76a5764 Fix variable pblk len, cleanup, rearrange
This is tested with the performance profile and works.

Lots of cleanup and rearranging also.
2019-11-13 15:13:12 -08:00
Tom Marshall 8229ef90bd Make pblk len variable, add format profiles, and cleanup 2019-11-12 23:49:36 +01:00
Tom Marshall efc83e0dc2 Rename [pl]blk_alloc to [pl]blk_used
Also a bit of minor cleanup.
2019-11-12 14:57:06 +01:00
Tom Marshall e70d283921 Add stats
In /sys/fs/compress/device-name:
	lblk_size,
	pblk_used/total,
	lblk_used/total,
	pbat_r/w,
	lbatpblk_r/w,
	lbd_r/w
2019-11-12 14:45:42 +01:00
Tom Marshall c4aabad212 Rename lbatpage to lbatpblk
This is more accurate, and will matter if pblk size is variable.
2019-11-11 21:07:11 +01:00
Tom Marshall ceb0eb3230 Make cbd check actually do a check and do some cleanup
* cbd check now works (ish).
 * Added full-check arg to cbd check, link with liblz4, libz.
 * Added libcbd/util.c with goodies like verbose, ask_user_bool.
 * Rework kernel side params, stats to separate from user space.
 * Lock kernel stats when updating.
 * Add lblk_used to stats.

... and probably some other forgotten things.
2019-11-11 20:48:46 +01:00
Tom Marshall 66eeec39f5 Add error flag to cbd_params, set it on write error 2019-11-09 17:02:53 +01:00
Tom Marshall bd51b7cb89 Flush lbd on a timer in lbdcache instead of the main module 2019-11-08 12:24:21 -08:00
Tom Marshall 68956f70e8 Minimize the amount of pbat reading to find free pblks
* Keep track of full pbats.
 * Allow specifying whether pbat must be non-full in pbatcache_get().
2019-11-06 14:58:04 -08:00
Tom Marshall a2e4f303fd Implement better caching
* Total cache size is the minimum of:
   - 1/1k of system RAM.
   - 1/64k of backing device.
   - 32 * 2 * CPUs.
   The latter ensures that we have at least 2*CPUs lbd objects
   which is required for the top level compress operations.

 * Implement LRU for all caches.

 * Track pblk full and last allocated block for better performance.

Still could use better parsing for cache_pages parameter and more
granularity, but this is workable for now.
2019-11-06 13:23:28 -08:00
Tom Marshall 3e81efb9f6 Defer releasing lbd until all I/O ops are done
This greatly increases performance.
2019-11-05 13:56:47 -08:00
Tom Marshall d7fb50911b Start implementing stats
Just pblks allocated for now.

Also update device layout info in README.
2019-11-04 12:25:54 -08:00
Tom Marshall f37155f527 Add flags to header
Initial flag definition includes dirty flag.

Also remove unused pblk_read().

Also add pbat-len arg to format.
2019-11-04 11:36:12 -08:00
Tom Marshall 141888fa98 Make pbat len variable
Also a bit of cleanup in lbd.c
2019-11-04 11:18:28 -08:00
Tom Marshall b8395c8a83 Alloc compress state using vmalloc 2019-11-03 09:00:28 -08:00
Tom Marshall 5be74b3346 Make pblk_read/write take block_device, not cbd_params 2019-11-02 08:11:12 -07:00
Tom Marshall 7fbf77b1f3 Revert "Add md5 for debugging"
This reverts commit 351e43b953.
2019-11-02 07:54:02 -07:00
Tom Marshall ee7eacd4a6 First really working version 2019-11-01 14:41:11 -07:00
Tom Marshall 164a09b9aa Add zlib support
* New args to cbd format.
 * HAVE_* -> COMPRESS_HAVE_*.
 * Verify header params algorithm and compression.
 * Move percpu alloc to lbdcache.
 * Alloc percpu data in lbdcache_ctr, free in lbdcache_dtr.
2019-10-31 11:09:03 -07:00
Tom Marshall eeafc209a5 Use per-cpu lz4 state 2019-10-30 12:34:21 -07:00
Tom Marshall f8361d1e2e First fully working version 2019-10-25 10:03:00 -07:00
Tom Marshall 351e43b953 Add md5 for debugging 2019-10-25 05:56:20 -07:00
Tom Marshall 446a4811f6 More improvements, but still failing to clone linux 2019-10-24 13:02:03 -07:00
Tom Marshall 495d191d16 checkpoint: Mostly working
This seems to work except for I/O timing.  Reads are sync and
writes are async, so this sequence fails:
  - Object x flushes
  - Object x is reused as y
  - Another object is taken as x
  - New object x reads
  -> Stale data is read

Two potential solutions from here:

1. Implement async reads.

2. Hold ref to object until write completes.

(1) is complicated, but more correct.  Writes may stay in buffers for
quite some time (typically 5 seconds), during which time the dm-compress
object cannot be released.
2019-10-21 19:39:27 -07:00
Tom Marshall 8ff29f0262 Rename [pl]blk_alloc to [pl]bat 2019-10-11 08:28:37 -07:00
Tom Marshall 07f4221a5f Fix a bunch of things
* Fix command line handling.
* Move option/parameter validation to libcbd.
* Make resize argument the logical size.
* Make blkdev_pblk_* take dm_compress instead of block_device.
* Reinstate dm_compress.io_failed.
* Other misc stuff.
2019-10-10 13:27:37 -07:00
Tom Marshall b675fe573a Pass buffer to elem rather than alloc in lblk_alloc_elem_{get,put}
This will assist in {read,writ}ing only the needed lblk alloc pages.
2019-10-09 11:57:57 -07:00
Tom Marshall 0ca9fc789f Rename some stuff and limit LBLK_SHIFT_MAX 2019-10-09 11:32:36 -07:00
Tom Marshall 2e18584af3 Initial working version
Slow but seems functional.
2019-10-09 11:04:40 -07:00