Johann was the illegitimate son of the clarinettist Kaspar Schrammel and his late wife Aloisia Ernst; his younger brother Josef Schrammel was in a similar situation. He also had an older half-brother named Konrad Schrammel (1833–1905), who had a less prestigious livelihood as a barrel organ player as he was forced to retire as an "invalid" from military service.

In his first musical lessons Schrammel got together with his brother, with assistance from his father. With approximately six years, Johann Schrammel could sing in the church choir in his home town Neulerchenfeld. His father had helped him receive violin lessons from Ernst Melzer despite a poor financial position.

On 6 January 1861, Johann Schrammel debuted along with his brother and father at a concert at the local inn "Zum goldenen Stuck" at the Neulerchenfelder Straße. From 1862 to 1863 he and his two brothers studied music at the Vienna Conservatory. In addition to singing lessons, the violin was taken up by Josef Schrammel.

In June 1865 Johann Schrammel left the Conservatory, and unsecured sources claim that he was at that time already a member of the orchestra of Danzers Orpheum, Harmonietheater.

In 1872 he married Rosalia Weichselbaumer and had 13 children with her; 9 of which survived childhood.

In 1878 he founded with his brother Josef Schrammel and Anton Strohmayer a trio; they were later joined by the clarinettist Georg Dänzer to form the famous Schrammel Quartet.

Johann Schrammel died at age 43 and was awarded an honorary grave in the Hernalser Cemetery.

• In 1923, in the district of Hernals, the Schrammelgasse (Schrammel Street) was named after him.

• In 1931, a commemorative plaque was placed at the home of the brothers in the Kalvarienberggasse 36.
• In 1932 the Elterleinplatz Alszauberbrunnen was built. Johann is sat on a stone monument as almost life-size figures. However, during the Second World War, the figures were melted down. In 1981, it was replaced by replicas of Heriberth Rath
• In 1967 a Schrammel monument was erected in a park by Dornbacher Street

• Der Schwalbe Gruß, op. 105. Lied, Text by Karl Lindau.
• Faschingskrapfen. Walzer.
• Glück und Liebe. Walzerlied, Text by Eduard Merkt.
• Hechten-Marsch.
• Kunst und Natur. Marsch.
• Praterveigerln, op. 151. Polka française.
• ’s Herz von an echten Weana. (Bekannt unter dem Namen „Schrammel Walzer“); Walzer für Piano mit Gesang ad libitum, Text by Carl Lorens.
• Vindobona, du herrliche Stadt. Walzerlied, Worte von Karl Schmitter.
• Warum da Weana in den Himmel kumma muass. Original-Couplet Eduard Merkt.
• Was Öst’reich is’. Alt-Wienerlied, lyrics by Wilhelm Wiesberg.
• Weana G′müat, op. 112. Walzer.
• Wien – Berlin, op. 100. Marsch.
• Wien bleibt Wien! Marsch.
• Wiener Künstler, op. 111. Marsch, text by C. M. Haslbrunner.

• Rudolf Alexander Moißl: Die Schrammel-Dynastie. St. Pöltner Zeitungs-Verlags-Gesellschaft, St. Pölten 1943.
• F. Mailer, H. Reitterer: "Schrammel Johann". In: Österreichisches Biographisches Lexikon 1815–1950 (ÖBL). Vol. 11, Austrian Academy of Sciences, Vienna 1999, ISBN 3-7001-2803-7, p. 171–173. (Direct links to "p. 171", "p. 172", "p. 173")
• Felix Czeike: Historisches Lexikon Wien, Band 5. Kremayr & Scheriau, Wien 1997, ISBN 3-218-00547-7, p. 141.

• Works by or about Johann Schrammel at Internet Archive
• Literature by and about Johann Schrammel in the German National Library catalogue
• Johann Schrammel in Austria-Forum (in German) (at AEIOU)
• „Die Schrammeln – Klassiker der Wiener Volksmusik"; Heinz Hromada über Werdegang und Musik der Brüder Schrammel
• Schrammelmusik oder Die Schrammeln. Was versteht man heute darunter?