Why does checking if temp table exists when "CONCAT_NULL_YIELDS_NULL" is set to ON doesn't work?

SET CONCAT_NULL_YIELDS_NULL OFF;  

IF OBJECT_ID ('tempdb..##QueryResults') IS NOT NULL
    DROP TABLE ##QueryResults;

Why is it that if i set set CONCAT_NULL_YIELDS_NULL to off as shown above and temp table ##QueryResults exists, it is successfully dropped but if i set it to ON, then temp table is not dropped even when it exists?