FastCFS v5.3.2 released

FastCFS V5.3.2 released, which mainly for bug fixes and minor improvements.

change logs:
[fstore] protocol ACTIVATE_SERVER do NOT response for RDMA
[fdir & fstore] bugfixed: MUST seperate pd for RDMA network
[fdir & fstore] check if the listening port matches the cluster configuration
[fstore] fix hash_code type from uint32_t to uint64_t in func fs_client_bs_operate
[fstore] return EINVAL when data group id NOT belongs to me
[FastCFS] fcfs_fused.c: connection pool stat for debug
[libfastcommon] connection pool performance optimization
[libserverframe] task init callback support extra argument
[libserverframe] struct sf_context add field is_client for RDMA callback alloc_pd
[libfastrdma] global init use CAS lock free for singleton

FastCFS 5.1 released

After one month of development and test, FastCFS V5.1 has been released, with major improvements including support for IPv6 and bug fixes.

The corresponding git log is as follows:
[fdir & fstore] auto support IPv6 by cluster.conf
[libdiskallocator] check store paths duplicate
[libdiskallocator] call crontab task trunk_index_dump with new thread
[libdiskallocator] trunk_index_dump optimize
[fstore] bugfixed: binlog_check.[hc] set replica's data group id correctly
[fstore] cluster.conf add item use_hash_func to disperse data group distribution

Welcome everyone to test and use V5.1. It is recommended that old users upgrade to the latest version. If you have any questions or suggestions, please feel free to join the group and communicate.

FastCFS 3.7.1 released

FastCFS 3.7.1 is released, supports user additional groups, and fully complies with POSIX standards. It solves the access rights problems that may occur when running NFS (dolly mode) on FastCFS's fuse client.

Improvement and fixed bugs are as follows:
[fdir] service_stat add fields: version, auth_enabled and storage_engine
[fdir] list dentry support special dentries . and ..
[fdir] call get_path_block_size only when read_direct_io is true
[fdir] dentry_remove bugfixed: correct posix ACL check
[fstore] service_stat add version field and support options: -G and all
[libserverframe] check socket connected on unexpected stage
[libserverframe] get_leader_connection: failover on multi ip addresses
[libserverframe] change log level to debug for hash entry reclaim
[libfastcommon] parse_bytes support space charactors (before and after the unit)
[libfastcommon] fc_fallocate fail back to ftruncate under Linux

FastCFS provides rpm and deb packages for intel and arm architectures respectively. Welcome to install and experience.

FastCFS 3.7 released

FastCFS 3.7 was released. It passed POSIX compatibility tests and fixed 7 bugs, so further improving the stability of FastCFS.

The POSIX compatibility test uses the open-source project pjdfstest. There are more than 8000 test cases in total, and only a few cases fail. Among them, 30 FIFO type file open cases fail. Such open calls will not be passed to the fuse layer, and the return value depends on the Linux kernel implementation.

The list of bugs fixed in v3.7:
[libfastcommon] bugfixed: common_blocked_queue_[alloc|free]_node must use lock
[libfastcommon] bugfixed: can't use global malloc_allocato
[fstore] bugfixed: skip rpc to slaves when no update for allocate space
[fstore] bugfixed: remove_conflict_slices add to two chains correctly
[fdir] bugfixed: define macro FDIR_FLAGS_FOLLOW_SYMLINK correctly
[fdir] bugfixed: normalize_path for symlink correctly
[fdir] return ENOLINK for readlink when path is not a symlink

FastCFS provides rpm and deb packages. You are welcome to install and experience them. Users who use the old version of FastCFS should upgrade to the latest version ASAP.

FastCFS v3.4 released

After a month and a half of intensive development and testing, v3.4 was released with following main improvements:

1. introduce vote node to prevent brain splitting in 2 data copies (i.e. dual active & standby & avoid brain splitting to ensure data consistency);
2. binlog dedup and historical data cleaning of fdir and fstore to reduce unnecessary disk space occupation;
3. fdir's file lock implemented strictly follows POSIX specification:

1. allow re-entry (repeatedly locking in the same process);
2. unlock returns success anyway;
3. one unlock can unlock many locks;
4. unlock supports partial unlocking.

4. remove unnecessary error logs and add the service name to the network connection failure log for problem location.

The fixed bugs are:
[fdir] bugfixed: must check load children for list dentry when storage engine is enabled
[fdir] bugfixed: CAS set data version correctly
[fdir] bugfixed: MUST change replication ptr array in the cluster thread
[fauth] bugfixed: padding with the terminating '\0' character for strtoll
[fstore] bugfixed: send active test when (task->offset == 0 && task->length == 0)

FastCFS v3.3 released

FastCFS V3.3 was released after a month and a half of intensive development and testing. V3.3 is a milestone version, the system stability and data consistency have been fully tested and guaranteed on normal restart and force kill by kill -9.

The main improvements are:

1. [fstore] fixed the data consistency problem in the case of server restart;

2. [fstore] single disk data recovery: after single disk fault recovery, data can be recovered through command line option;

3. [fstore] master rebalancing: after the recovery of machine failure or short-term network failure, the master is rebalanced;

4. [fauth & fdir & fstore] introduce anti brain-split mechanism: the principle of more than half voting is adopted in leader / master election.

The list of fixed bugs as follows:

1. bugfixed: done_bytes must use atom decrease under multi threads

2. bugfixed: DG master election stopped in rare case

3. bugfixed: MUST waiting slice write done before trunk reclaim

4. bugfixed: should keep delete-slice binlog when ob_index_get_slice_count > 0

5. timestamp in the replica and slice binlogs increases monotonically

6. bugfixed: should scan last N + 1 seconds binlog when binlog check and repair

7. bugfixed: log replica binlog by the fetched binlog

FastCFS V3.3 has met the requirements of the production environment. You are welcome to experience and test it.

FastCFS v3.2 released

After one and a half months of development and testing, FastCFS v3 2.0 launched. This version mainly improves:

1. Provide a full POSIX APIs such as open/fopen, write/fwrite, read/fread, close/fclose, readv, writev, etc., through which applications can use FastCFS storage;

2. use LD_PRELOAD to realize virtual mount point in user space;

3. In order to support POSIX API, fdir has been adjusted and improved. For example, list/remove dentry and get/list/remove xattr support flags; libfastcommon and fstore support readv and writev;

4. fstore supports clearing redundant binlog data after cluster expansion (add option --migrate-clean when starting fs_serverd).

Other improvements and bug fixed as follows:
[fastcfs-csi] config optimization
[fstore] read ahead support prefetch automatically
[fuseclient] fuse.conf add parameter: xattr_enabled
[fdir] bug fixed: update field "mode" correctly
[fdir] bug fixed: server_parse_inode_for_update MUST set dentry_type

FastCFS POSIX API introduction please see Chinese blog：https://blog.csdn.net/happy_fish100/article/details/123394587

FastCFS v3.1 released

FastCFS V3.1 mainly improves the core component FastDIR, realizes the LRU elimination algorithm, and supports massive files with limited memory.

The elimination algorithm of distributed directory service FastDIR has two characteristics:

  1. Eliminate by directory structure: first eliminate child nodes, then eliminate parent nodes;
  2. Eliminate by data thread: each data thread is an independent unit, and data access and 
     elimination are completed in it‘s data thread by lock free.

Other improvements and bug fixes are as follows:

  1. The reference counting method is adopted to release dentries without delay;
  2. Delay free namespace string for lockless;
  3. Bug fixed: get trunk_file_size from ini file correctly.

You are welcome to download and experience FastCFS. If you have any questions during installation and use, please feedback at any time.

FastCFS V3.0 released, supporting 10 billion of massive files

After five months of dedicated development, FastCFS 3.0 was finally released. Main improvements of FastCFS 3.0: FastDIR which is the core component, implements the data storage engine through plugin. Binlog + storage engine plugin is used to load inode data on demand. The single machine supports 10 billion massive files with limited memory (such as 64GB).

It is easy to realize data persistence through binlog. When the program restarts, all inode data is loaded into memory through binlog replay. There are two problems in storing massive files in this way:

  1. Long time for program startup readiness;
  2. Very high requirements for memory.

V3.0 introduces the storage engine plugin, which solves the two major problems of data persistence only through binlog. The following technical articles will introduce the principle and characteristics of FastDIR storage engine in detail. Please look forward to it.

In addition, FastCFS 3.0 fixes the following three bugs:

   [fdir] increase/decrease parent's nlink on rename operation
   [fdir] set dentry->kv_array->count to 0 correctly
   [fstore] should init barray->count to 0

Welcome to install and use the latest version of FastCFS. If you have any questions and suggestions, please feel free to communicate.