Global-scale studies indicate that semiarid regions strongly regulate the terrestrial carbon sink. However, we lack understanding of how climatic shifts, such as decadal drought, impact carbon sequestration across the wide range of structural diversity in semiarid ecosystems. Therefore, we used eddy covariance measurements to quantify how net ecosystem production of carbon dioxide (NEP) differed with relative grass and woody plant abundance over the last decade of drought in four Southwest U.S. ecosystems. We identified a precipitation “pivot point” in the carbon balance for each ecosystem where annual NEP switched from negative to positive. Ecosystems with grass had pivot points closer to the drought period precipitation than the predrought average, making them more likely to be carbon sinks (and a grass-free shrubland, a carbon source) during the current drought. One reason for this is that the grassland located closest to the shrubland supported higher leaf area and photosynthesis at the same water availability. Higher leaf area was associated with a greater proportion of evapotranspiration being transpiration (T/ET), and therefore with higher ecosystem water use efficiency (gross ecosystem photosynthesis/ET). Our findings strongly show that water availability is a primary driver of both gross and net semiarid productivity and illustrate that structural differences may contribute to the speed at which ecosystem carbon cycling adjusts to climatic shifts.