The self-proclaimed toughest sheriff in America, Joe Arpaio, has become a poster child for tough illegal immigration enforcement in the United States.

His 20 years in office have been defined by his emphasis on the border, and his push to unceasingly crack down on crime.

This year however, the Maricopa County sheriff is in the middle of the most difficult re-election fight of his career — largely because those themes are being turned against him.

Don’t blame me. I’m just the sheriff…I’m just doing my job, I’m not a racist.

- Sheriff Joe Arpaio, Maricopa County

Arpaio has been dogged by revelations that his office failed to adequately investigate more than 400 sex-crimes cases — including dozens of alleged child molestations — and allegations that his deputies have racially profiled Latinos in his boundary-pushing immigration patrols.

He told Fox News Latino from the Republican National Convention, in his famously unvarnished style, that he’s not the root of any alienation between Latino voters and the Republican Party caused by immigration rhetoric.

“Don’t blame me,” he says. “I’m just the sheriff…I’m just doing my job, I’m not a racist.”

It has all added up to a tough fight for metro Phoenix’s 80-year-old sheriff, who has become a national political figure on the immigration issue and was courted by Republican presidential candidates this year as they sought to bolster their anti illegal immigration credentials.

A lack of reliable independent polling makes it hard to tell how competitive the race is. Arpaio, for his part, has far more money on hand — $3.8 million at last count — than his rivals. Arpaio's campaign has been fueled by donors outside of Arizona - who have accounted for 80 percent of his $8 million campaign fund.

While this isn't new for Arpaio, some like Bruce Meril, a longtime Arizona political scientist and pollster, say he's also fueled out-of-state donations against him.

"Joe's become the focus of the Senate Bill 1070 groups. Since (ousted Arizona Senate President) Russell Pearce is now gone, Joe is really kind of the face of that whole movement," Merrill told the Arizona Republic. "This race is just heavily influenced by that, and the money that the sheriff has been able to raise may well keep him in office."

David Berman, a senior research fellow at Arizona State University’s Morrison Institute for Public Policy, said it’s unclear whether Arpaio will prevail again, but he’s more vulnerable than ever.

Berman said some voters disapprove of his handling of sex-crimes cases and are tired of his focus on illegal immigration. “There are people who are concerned, who want to get away from the state’s kooky image,” Berman said. “Sometimes, he feeds into that.”

The five-term sheriff said he has dealt with sex-crimes cases responsibly once they were brought to his attention, denied the racial profiling allegations and offered few regrets for his management of the agency. “I am not changing,” Arpaio said in a recent interview at a Denny’s where several diners walked up to greet him.

Despite tougher competition, Arpaio remains popular with many voters who are frustrated with the porous Arizona-Mexico border and approve of his practice of making jail inmates wear pink underwear and sleep in canvas tents in the sweltering desert heat.

Arpaio faces a challenge from Democrat Paul Penzone and Independent Mike Stauffer, who have hammered on the problems with sex-crimes investigations. “He does not want to be held accountable,” said Stauffer, a 51-year-old retired Scottsdale officer. “He’d rather we’d forget about it.”

The sheriff’s office reopened more 400 sex-crimes cases that were reported to the agency but were inadequately investigated or not investigated at all after they were reported over a three-year period that ended in 2007.

A city that had contracted with the sheriff’s office for police services had concluded there were many cases in which sheriff’s investigators wrote no follow-up reports, collected no additional forensic evidence and made zero effort after the initial report of the crime was taken. The city later concluded that some cases were no longer viable, in part, because victims had either moved away or otherwise moved on.

Arpaio’s office said it has moved to clear up inadequately investigated cases and has taken steps to prevent the problem from happening again, such as doubling the number of detectives on the sex-crimes squad, increasing training and setting up a system that alerts supervisors if there has been no activity in a case within 30 days.

The sheriff said reporters have “reinvented” a five-year-old story about the sex-crimes squad and his agency has been singled out when several other police departments in metro Phoenix have had similar problems.

County prosecutors who reviewed nearly 500 sex-crimes cases from Arpaio’s office had concluded two-thirds of the cases were handled properly, but they still would have never made it to court because there was no crime committed or no corroborating evidence.

Prosecutors concluded further investigation wasn’t possible in about 12 percent of cases because there were no medical exams conducted on victims, officers failed to follow up after a crime was reported or other deficiencies. Victims in another 11 percent of cases didn’t want to follow through with prosecutions. The sheriff’s office is expected to resubmit 28 cases for possible prosecution.

Penzone, a 45-year-old retired Phoenix police officer, said Arpaio gives short shrift to the duties he was hired to do and focuses instead on operations that elevate his public profile.

“He neglects fundamental responsibilities because they lack the sensationalism that draws media attention,” said Penzone, who ran a commercial on the bungled cases that questions Arpaio’s reputation for getting tough on crime.

The sheriff, in turn, launched an ad accusing Penzone of abusing his then-estranged wife in 2003, an allegation the challenger denied. No charges were filed. Penzone said it was his wife who hit him in the head during the dispute with a hockey stick.

A figurative thorn in Arpaio’s side was removed in late August when federal prosecutors said they were closing an abuse-of-power investigation against the sheriff’s office without filing charges. The investigation focused on the investigative work of the sheriff’s anti-public corruption squad that brought failed cases against officials who were at odds with him.

While the sheriff’s immigration enforcement has made him popular among some voters, it has led to two lawsuits that allege racial profiling in his immigration patrols. A federal judge heard one of the lawsuits this summer and hasn’t yet issued his ruling. The lawsuit filed by a small group of Latinos will serve as a precursor for a similar yet broader civil rights lawsuit filed against Arpaio by the U.S. Department of Justice.

Arpaio disputed claims that he’s facing his toughest campaign. He has sought to deflect the criticism by running ads promoting his crackdowns on deadbeat parents and animal abusers. Still, he conceded he’s catching more heat over his immigration enforcement this time around.

“There is more anger out there,” the sheriff said.

Reporting by the Associated Press.

Follow us on twitter.com/foxnewslatino
Like us at facebook.com/foxnewslatino