Adrian Ramos scored in the 45th minute, finally breaking through after Costa Rica's Randall Brenes was sent off in the 28th on a straight red card.
Colombia leads Group A from Bolivia and Argentina following their 1-1 draw on Friday in the tournament opener. Costa Rica, playing as a guest team in the South American championship, is without a point and is fielding a youth team, as is Mexico.
''To start with a victory always gives you confidence,'' Ramos said. ''It wasn't a clear victory, but the important thing is that we won.''
Colombia was unimpressive, seldom dominating a short-handed team featuring mostly young, inexperienced players. Colombia could get a shock when it faces Argentina on Wednesday with the Gauchos under pressure at home after a disappointing opening match.
Brenes was sent off after a hard tackle, but Colombia struggled to take advantage.
Ramos broke through on an individual effort. Taking a through pass, he raced into the area and then dribbled around Costa Rica goalkeeper Leonel Moreira before slotting home from short range.
Colombia star forward Falcao Garcia seldom had the ball in position to strike and went off in the 77th. He had his best chance in the 57th, but his header struck the crossbar.
Colombia coach Hernan Dario Gomez said this week he would resign if the team failed to reach the quarterfinals. Despite a poor game, the victory was a big step for Colombia to reach the second round.
Costa Rica's Argentina-born coach Ricardo La Volpe said he was already looking ahead to Thursday's match against Bolivia.
''We have to win this game,'' he said. ''We're going to work 30 percent on defense and 70 percent on attack.''
Costa Rica was named as a late entry in the tournament after Japan withdrew following the March 11 earthquake and tsunami.
Brazil has won four of the last five titles, and defeated Argentina in the last two finals.
Colombia: Luis Martenez, Camilo Zuniga, Luis Amaranto Perea, Mario Yepes, Pablo Armero, Gustavo Bolivar, Abel Aguilar (Hugo Rodallega, 34), Freddy Guarin, Adrian Ramos, Dayro Moreno (Teofilo Gutierrez, 69), Radamel Falcao Garcia (Elkin Soto, 76).
Costa Rica: Leonel Moreira, Oscar Duarte, Jonhnny Acosta, Francisco Calvo, Jose Salvatierra, David Guzman (Jose Cubero, 74), Pedro Leal, Heiner Mora, Randall Brenes, Diego Madrigal (Josue Martinez, 71), Joel Campbell (Cesar Elizondo, 45).