An efficient, lossless storage system (DEHYDRATOR) for the provenance graphs that security analysts build from operating-system audit logs. It removes redundancy at two levels, field mapping encoding for repeated field values and hierarchical encoding for repeated graph structure, and trains a deep neural network to regenerate log sequences in place of large calibration/lookup tables, so the stored graph can still be batch-queried. It preserves all original data (avoiding the false negatives of pruning-based methods) while reducing storage by roughly 85% and greatly outperforming general-purpose graph and relational databases and prior encoding systems.
Security analysts investigate cyber attacks using provenance graphs built from operating-system kernel audit logs, where nodes are system entities (processes, files, sockets) and edges are events. Be…