Downie began his junior career after being selected in the first round with the sixteenth overall pick in the 2003 OHL Priority Draft by the Windsor Spitfires. As a rookie in 2003-2004, he had 7 goals and 16 points with the Spitfires. In 2004-2005, at the age of 18, he propelled himself into first round draft contention with an outstanding 21 goals and 73 points in in 61 games with Windsor. He finished the year rated the thirty-eighth best North American skater prospect by NHL Central Scouting for the 2005 NHL Entry Draft, and was eventually selected twenty-ninth overall in the 2005 NHL Entry Draft by the Philadelphia Flyers. Unfortunately, the 2005-2006 season was the beginning of Downie's infamous reputation as a player with disciplinary problems. At one point during the season, he was suspended for an on-ice incident during practice where he attacked and fought a teammate for refusing to take part in a hazing ritual. The incident eventually cost Windsor's head coach his job and led to Downie being traded to the Peterborough Petes. He ended up with 19 goals and 53 points in 35 games to go along with 113 penalty minutes in his time split between Windsor and Peterborough. Despite the incident, the Flyers signed Downie in the Fall of 2006 before returning him to junior, where he again put up outstanding numbers finishing with 35 goals and 92 points in just 45 games to go along with 124 penalty minutes in time split between Peterborough and Kitchener, where he was traded at the deadline in 2007. In 2006, Downie also made his first appearance for Team Canada, helping the team win a gold medal in the 2006 U20 World Junior Championships.
In 2007-2008, Downie began his professional career full time. He played 21 games for the Philadelphia Phantoms of the AHL scoring 5 goals and 17 points in 21 games. He also appeared in 32 games for the Flyers at the NHL level, scoring 6 goals and 12 points in 32 games. The Flyers also allowed Downie to play in the 2007 U20 World Junior Championships where he again helped Canada with the gold medal. On paper, it was another good season for Downie, but it was again marred by a suspension stemming from a preseason game where Downie boarded Ottawa's Dean McAmmond, resulting in a 20 game suspension. In 2008-2009 Downie was traded to the Lightning in a deal involving defensemen Steve Eminger and Matt Carle. Downie was scoreless in 6 games with the Flyers and had 1 goal and 8 points in 4 games with the Phantoms in the AHL prior to the trade. After the trade, Downie appeared in 23 games with Norfolk of the AHL, scoring 8 goals and 25 points to go along with 107 penalty minutes. He also appeared in 23 games with the Lightning scoring 3 goals and 6 points with 54 penalty minutes. Unfortunately, Downie again earned a 20 game suspension in the second half of the season in a game with Norfolk in which he deliberately slashed an official in the shins on a faceoff.