The group is protesting because they believe ethnic violence and human rights abuses under Modi have put India on a genocide watch list.
Since he took office five years ago, the group said violence against social and religious minorities has become much worse.
"There are 800,000 military on the Kashmir," protester Syed Alie said. "Their people are denied their access to freedom. They cannot get out of their home."
Protesters said children are being incarcerated, people are being tortured by the police and shot at by Indian Troops and there is an Internet and cell phone blackout in Kashmir.
The protesters gathered on the south side of NRG, chanting, "Go back Modi" and "Modi is a terrorist." They said they wanted to make sure Modi knew he was not welcome in Houston.
"America is not the place for Modi," Protester Gurpatwant Pannun said. "America stands for the right to life and liberty and property and that is exactly what Modi stands against."
Their chants could be heard down the streets, but they were not hopeful that Modi was going to get their message.
"Modi is deaf and blind any way, because if he can see something, he already sees he is killing eight million Kashmirians," Pannun said.
READ MORE: Thousands to greet world leaders on Houston stage
[Ads /]
About 50,000 people were expected to attend "Howdy, Modi" at NRG.
Houston and India are business partners. You may remember Texas Gov. Greg Abbott meeting with Prime Minister Modi while traveling to India last year.
[Ads /]
Houston-India trade is worth more than $7 billion annually. Dozens of companies have a presence in Houston and in India.
Follow Mycah Hatfield on Facebook and Twitter.