Causal Discovery Algorithms for Extracting Latent Structural Relationships in Deep Models

Abstract

Deep Neural Networks have greatly impacted machine learning technology, but the inner workings of such systems remain mysterious, making trust and interpretability difficult. The ability to discover the causal relations inside these systems is vital in enhancing understanding, reliability, and robustness. This paper proposes a new technique for discovering hidden structural relationships inside deep neural networks called CDAL (Causal Discovery in Deep Learning). This novel technique involves merging graphical causal modeling with the analysis of deep learning techniques. Information theory is used to reveal causal relationships between different neurons and layers inside deep neural networks. The approach includes three phases where the first phase, Causal Graph Construction, utilizes Granger Causality and Convergent Cross-Mapping, which is tailored for neural activations. The second phase is Latent Relation Extraction, which involves independence testing and constraint-based techniques. Lastly, Structural Interpretation utilizes causal inference methods on the generated graph for gaining mechanistic insights into the workings of the network. Studies indicate that CDAL obtains 89% accuracy in recovering the true causal relations from a synthetic network and discovers meaningful causal relations within trained neural networks used for tasks such as image classification and natural language processing. The discovered relations reveal information regarding how features learn, information flows, and possible weaknesses within deep networks. Also, models having causal structures prove to be 23% more robust against adversarial attacks and generalize better on out-of-distribution data. By combining the concepts of explainability and mechanistic insights, new possibilities arise in the domain of model debugging, safety verification, and neural computation analysis.