Incentivized advertising is a new ad format that is gaining popularity in digital mobile advertising. In incentivized advertising, the publisher rewards users for watching an ad. An endemic issue here is adverse selection, where reward-seeking users select into incentivized ad placements to obtain rewards. Adverse selection reduces the publisher's ad profit as well as poses a difficulty to causal inference of the effectiveness of incentivized advertising. To this end, we develop a treatment effect model that allows and controls for unobserved adverse selection, and estimate the model using data from a mobile gaming app that offers both incentivized and non-incentivized ads. We find that rewarding users to watch an ad has an overall positive effect on the ad conversion rate. A user is 27% more likely to convert when being rewarded to watch an ad. However there is a negative offsetting effect that reduces the effectiveness of incentivized ads. Some users are averse to delayed rewards, they prefer to collect their rewards immediately after watching the incentivized ads, instead of pursuing the content of the ads further. For the subset of users who are averse to delayed rewards, the treatment effect is only 13%, while it can be as high as 47% for other users.