Many applications of the level-k model assume that individuals employ a fixed depth of reasoning across different games. To study the validity of this assumption, our subjects play games designed to identify inconsistent depth of reasoning without relying on the results of an econometric model. Most subjects’ choices are not consistent with a fixed depth of reasoning even in extremely closely related games. The likelihood of inconsistency increases with cognitive ability, suggesting that it is not solely due to confusion. Higher optimization premiums are correlated with greater depth of reasoning, but do not reduce the likelihood of inconsistency.