Genome-wide association studies find that a large number of genetic variants jointly influence the risk of depression, which is summarized by polygenic indices (PGIs) of depressive symptoms and major depression. But PGIs by design remain agnostic about the causal mechanisms linking genes to depression. Meanwhile, the role of adverse life experiences in shaping depression risk is well-documented, including via gene-environment correlation. Building on theoretical work on dynamic and contingent genetic selection, we suggest that genetic influences may lead to differential selection into negative life experiences, forging gene-environment correlations that manifest in various permutations of depressive behaviors and environmental adversities. We also examine the extent to which apparent genetic influences may reflect spurious associations due to factors such as indirect genetic effects. Using data from two large surveys of middle-aged and older US adults, we investigate to what extent a PGI of depression predicts the risk of 27 different adversities. Further, to glean insights about the kinds of processes that might lead to gene-environment correlation, we augment these analyses with data from an original preregistered survey to measure cultural understandings of the behavioral dependence of various adversities. We find that the PGI predicts the risk of majority of adversities, net of class background and prior depression, and that the selection risk is greater for adversities typically perceived as being dependent on peoples’ own behaviors. Taken together, our findings suggest that the PGI of depression largely picks up the risk of behaviorally-influenced adversities, but to a lesser degree also captures other environmental influences. The results invite further exploration into the behavioral and interactional processes that lie along the pathways intervening between genetic differences and wellbeing.