So, What am i doing wrong? (stealth and infiltration)

Not sure if youve done it this way but If you have scouted first and spoken to sir radzig and then go back to sabotage the camps on high alert and makes it alot harder also

It also has alot to do with the decisions you made to get to that point. If you helped the bandits find timmy youd just be able to walk around