Skip to main content

SHIELD

Overview​

SHIELD is a research project that adds transparent encryption to RocksDB, a widely used log-structured merge-tree (LSM) key-value store, using SST Auth as the key management service. SHIELD covers three storage architectures: monolithic (local), disaggregated (HDFS-backed), and offloaded-compaction deployments.

Publication: Viraj Thakkar*, Dongha Kim*, Yingchun Lai, Hokeun Kim, and Zhichao Cao, "SHIELD: Encrypting Persistent Data of LSM-KVS from Monolithic to Disaggregated Storage," in Proc. ACM Conference on Management of Data (SIGMOD), Berlin, Germany, June 22–27, 2025. (* Equally contributed lead authors.) DOI: 10.1145/3725354

Repository: iotauth/SHIELD
Language: C++ (80%), Java (12%), C, Python, Shell

Motivation​

As key-value stores are increasingly deployed in disaggregated and cloud storage environments, protecting persistent data at rest becomes critical. SHIELD integrates SST Auth as a Key Distribution Center so that encryption keys for database files are managed, rotated, and authorized through the same mechanism used for entity-to-entity secure communication — providing uniform key governance across the SST ecosystem.

Architecture​

SHIELD adds an encryption layer to RocksDB at the SSTable (.sst) and WAL file level. Encryption keys are provisioned via SST Auth and stored in the RocksDB options path under iotauth/. Three deployment modes are supported:

ModeStorage backendCompaction location
MonolithicLocal filesystemLocal
DisaggregatedHDFS (remote)Local
Offloaded compactionHDFS (remote)Remote compaction server

Prerequisites​

  • OpenSSL
  • Java 11 (with JAVA_HOME set)
  • Maven
  • Node.js
  • RocksDB build dependencies (cmake, build-essential, libsnappy-dev, etc.)
  • Disaggregated / offloaded only: gRPC, HDFS

Install on Ubuntu:

sudo apt-get install -y openjdk-11-jdk maven nodejs npm \
    openssl cmake build-essential libsnappy-dev \
    libgflags-dev libbz2-dev liblz4-dev libzstd-dev

Monolithic installation​

1. Clone and build iotauth (Auth server)​

git clone https://github.com/iotauth/iotauth.git
cd iotauth
git submodule update --init
cd examples && ./generateAll.sh
cd ../auth/auth-server && mvn clean install

2. Clone and build SHIELD​

git clone https://github.com/iotauth/SHIELD.git
cd SHIELD
mkdir build && cd build
cmake -DWITH_SST_ENCRYPTION=ON \
      -DIOTAUTH_KEY_PATH=/path/to/keys \
      ../
make -j$(nproc)

3. Run a benchmark​

Unencrypted baseline:

./db_bench --benchmarks=fillrandom --num=1000000

With SHIELD encryption:

./db_bench --benchmarks=fillrandom --num=1000000 \
    --compression_type=encrypt \
    --wal_compression=encrypt

With EncFS filesystem-level encryption (alternative):

./db_bench --benchmarks=fillrandom --num=1000000 \
    --compression_type=encfs

Disaggregated storage installation​

Deploy HDFS on a separate server, then build SHIELD with the HDFS plugin:

cmake -DWITH_SST_ENCRYPTION=ON \
      -DWITH_HDFS=ON \
      -DHDFS_HOME=/path/to/hadoop \
      ../
make -j$(nproc)

Run with HDFS storage backend:

./db_bench --benchmarks=fillrandom \
    --compression_type=encrypt \
    --db=hdfs://namenode:9000/rocksdb-data

Offloaded compaction installation​

Build on both the coordinator node and the remote compaction server, configuring the compaction address:

cmake -DWITH_SST_ENCRYPTION=ON \
      -DWITH_HDFS=ON \
      -DREMOTE_COMPACTION_ADDR=compaction-server:port \
      ../

IoTAuth integration​

The iotauth/ module inside SHIELD handles key provisioning:

  • On database open, SHIELD contacts SST Auth to obtain a session key for the current database instance.
  • SSTable files and WAL segments are encrypted with the provisioned key using AES.
  • Key rotation is managed through Auth's session key expiration and renewal workflow.

This means SHIELD databases participate in the same authorization and key-distribution infrastructure as SST entities: the same Auth instance, communication policies, and key governance apply to both network communication and data-at-rest encryption.

Repository structure​

SHIELD/
├── iotauth/            SST Auth integration (key provisioning)
├── include/rocksdb/    RocksDB public headers
├── db/                 Core database implementation (with encryption hooks)
├── table/              SSTable management
├── tools/              Benchmarking tools (db_bench)
├── examples/           Example configurations
├── docs/               Additional documentation
├── CMakeLists.txt      Build configuration
├── README.md           Full installation guide
└── INSTALL.md          Detailed installation instructions

Further reading​