The report released Thursday found that Arpaio's office carried out a blatant pattern of discrimination against Latinos.
"Our case is more than just statistics. Its about real people, law abiding residents of Maricopa County caught up in the web of unconstitutional activity and unlawfully stopped, detained and arrested," said Thomas Perez, Asst. U.S. Attorney General.
The report says Arpaio's office also held a "systematic disregard" for the Constitution amid a series of immigration crackdowns that have turned the lawman into a prominent national political figure.
At a news conference, Arpaio says the government's move will result in releases from jail of illegal immigrants who had previous been arrested for crimes.
They called it a Pearl Harbor-style sneak attack, nothing but a phony show of political pandering by the Obama administration, pointing to the lack of specifics in the 22-page report.
"President Obama and the band of his merry men might as well erect their own pink neon sign at the Arizona-Mexico border saying 'Welcome all illegals to your United States, our home is your home,'" said Sheriff Joe Arpaio.
He vows he will never resign, and is committed to running his department. He did acknowledge that there may have been some individual cases where deputies didn't do things by the book, and that he would look at those cases individually -- and rejected the sweeping generalizations made in the report.
The government found that Arpaio's office committed a wide range of civil rights violations against Latinos, including unjust immigration patrols and jail policies that deprive prisoners of basic Constitutional rights. They interviewed 400 people -- inculding 75 current and former employees of MCSO -- and Sheriff Arpaio himself.
"I do have compassion, but I'll tell you one thing. Enforcing the law overrides my compassion," Arpaio said.
The Justice Department wants the office to change its ways or risk losing federal money.
Arpaio has 60 days to implement changes, and then there will be a review. If progress has not been made by then, legal action could be taken.
Also Thursday, the Department of Homeland Security cut ties with MCSO because of the report, ending an agreement that allowed trained sheriff's deputies to enforce immigration law, and restricting the sheriff's ability to screen for inmates immigration status while they are in his jails.
A criminal investigation is still ongoing into abuse of authority allegations.
For more visit myFOXphoenix.com