Hundreds of thousands desperate refugees escape evil ISIS: can they organize opposition?

Apologies to all who find my opinions too simplistic, it just seems to me that hundreds of thousands of refugees outnumber 30,000 to 40,000 ISIS evil extremist terrorists.  

Although money is a root motivator of the evil ISIS movement, (somehow the group is raising big revenue) the fact is, those who are brutally persecuted by the barbaric terrorists outnumber the military extremists.

When hundreds of thousands of women and innocent civilians are driven into desperate refugee camps in Jordan, and now Turkey as well as other places, it seems like an organized effort among those huge numbers of people could turn their energies towards marching on the enemy. In Syria, the enemy is President Assad. In Iraq, the enemy is clearly the black flags marking the ISIS terrorist camps. 

When refugees, who number in the hundreds of thousands, are facing a future filled with desperation, it seems there's less to loose by facing down the enemies, rather than remaining in a limbo status, with little hope of returning to their homes. 

ISIS is estimated to be a movement consisting of about 20,000 to 40,000 extremist evil militants. Refugees fleeing from ISIS number in the hundreds of thousands of people who have no future as long as their existence is threatened. It just seems like ordinary math would motivate the refugees to face down the terrorist enemies, calling on the aid of Arab nations in the process, rather than allow their desperate situations to deteriorate.

 A coalition of nations is developing an anti extremist ISIS strategy. Does it make sense to arm the desperate refugees who have valid reasons to destroy the group that's taken away their homes? 

Yes it does.