Effective immunity requires a complex network of cellular and humoral components that interact with each other and are influenced by different environmental and host factors. We used a systems biology approach to comprehensively assess the impact of environmental and genetic factors on immune cell populations in peripheral blood, including associations with immunoglobulin concentrations, from similar to 500 healthy volunteers from the Human Functional Genomics Project. Genetic heritability estimation showed that variations in T cell numbers are more strongly driven by genetic factors, while B cell counts are more environmentally influenced. Quantitative trait loci (QTL) mapping identified eight independent genomic loci associated with leukocyte count variation, including four associations with T and B cell subtypes. The QTLs identified were enriched among genome-wide association study (GWAS) SNPs reported to increase susceptibility to immune-mediated diseases. Our systems approach provides insights into cellular and humoral immune trait variability in humans.