The killer - reported to be a 17-year-old former pupil - struck in the idyllic historical town of Winnenden in the Rems-Murr district.
Witnesses described a young person wearing black combat gear storming the Albertville School at 10am.
Without a word, he opened fire in corridors and rooms. Ten people have been reported to have been killed, with several more injured. Police confirmed the death toll and said there had been "numerous wounded".
A spokeswoman for the interior ministry in the state of Baden-Wuerttemberg said: "We have to assume a death toll that’s in the double-digits." She added: "These are students."
The gunman fled following the massacre and is said to be on the loose in the city centre. Police have evacuated the school and closed off the town and the surrounding regions to search for the killer, with helicopters circling above the town. Motorists have been warned against picking up hitchhkers.
"Police are coming through the whole time. They’re obviously looking all over town for him,“ said Roberto Seifert, who works at a company neighbouring the school. "We’ve never had anything like this,“ he told Reuters by phone.
Reports suggest the killer was a former pupil at the school, and his parents apparently had 18 weapons at home legally. Whether the gunman used weapons from their arsenal is not yet known.
In April 2002, Germany was rocked by its worst school shooting after a gunman killed 17 people, including himself, at a secondary school in the eastern city of Erfurt.
Today's tragic massacre comes after at least 11 people were killed in a shooting spree when a gunman went on the rampage in Alabama yesterday.