U.S. officials say the trade agreement with Colombia should be in place by May 15.

The Obama administration cited steps taken by the Colombian government to prevent violence against labor union members and improve workers' rights as a key reason the agreement finally can take effect.

The U.S. and Colombia agreed to the trade deal last year. But implementation of the pact was contingent on Colombia enacting reforms to protect unions.

The free trade agreement has wide support in the U.S. business community. But many unions who otherwise support Obama have been critical of the trade deal, saying that Colombia still has an abysmal record of violence against labor union members.

Now, President Obama is returning from the summit to his familiar agenda of righting the U.S. economy and winning a second term, and dealing with the Secret Service scandal.

Summit of the Americas: Unfulfilled Promises

Domestic issues are immediately on tap, with the Senate scheduled to vote Monday on Obama's proposal to increase taxes on millionaires. 

The proposal stands little chance of passing Congress, but Obama has cast it as an election-year theme as he seeks to paint sharp contrasts between himself and his likely Republican challenger, Mitt Romney.

Obama returned to Washington late Sunday with a key free trade deal with Colombia ready to be fully enforced next month and with important face time with Latin American leaders that cannot hurt his diplomatic outreach.

But the weekend trip to Cartagena, Colombia, for the sixth Summit of the Americas also underscored old and new fissures that exist between the United States and its southern neighbors, from the U.S. isolation of Cuba to calls by some Latin American leaders to defang the violent drug cartels by legalizing drugs.

The trip was clouded by unseemly allegations against Secret Service personnel and military service members working on security in Cartagena ahead of Obama's arrival. Obama, at a press conference in Cartagena, said that if the accusations, proved true "of course I'll be angry."

Nude Feud Divides Mexicana Flight Attendants Turned Calendar Girls

The Secret Service sent 11 agents home and placed them on leave for misconduct as the agency investigates what happened. Five members of the military working with the Secret Service were confined to quarters, pending an investigation into an alleged prostitution scandal.

"I expect that investigation to be thorough, and I expect it to be rigorous," Obama said. "We are representing the people of the United States, and when we travel to another country, I expect us to observe the highest standards."

The story could also be kept alive in Congress where at least one Republican committee chairman suggested the scandal may not be an isolated incident.

Obama began moving forward to domestic issues even as he was still wrapping up business in Cartagena. At the news conference, with Colombian President Juan Manuel Santos at his side, Obama mounted a vigorous defense of his tax proposals.

Mexico City Finds That Grit is Good for Tourism

"I want everybody to remember, I'm going to say this repeatedly: This is not an argument about taking from A to give to B. This is not a redistributionist argument that we're making. We're making an argument about how do we grow the economy so that it's going to be prospering in this competitive 21st century environment," Obama said.

Based on reporting by the Associated Press.

Follow us on twitter.com/foxnewslatino
Like us at facebook.com/foxnewslatino