During Aegon's Conquest, the village fell under attack, but was saved by Aegon the Conqueror himself. For this, the leader of the colony swore an oath to Aegon that they would fight and defend his bloodline for the rest of their days. They took the name the "Knights of Aegon", with their leader being known as "the Master", a hereditary title passed down from father to son.
Eventually, Gyllor Paenion became the Master of the Knights of Aegon by the time of the outbreak of Robert's Rebellion. They once again fought for House Targaryen but were unaware of Aerys II Targaryen's mental instability. They participated in the Siege of Storm's End and in some of the battle in the Borderlands.
After Robert's Rebellion ended, Gyllor informally disbanded the group when he learned that Aerys and his heir, Prince Rhaegar Targaryen, had both been killed during the war. However, when he learned that some other Targaryen children has survived, he joined the Order of the Dragon and made a promise to reform the Knights when the time came for one of them to sit on the Iron Throne.
When the War of the Five Kings had broken out, Gyllor had distanced himself from the Order of the Dragon's affairs in an attempt to live out the rest of his life in peace and solitude. Ultimately, he would go on to train Sarina Waters, a bastard daughter of Robert Baratheon into becoming a vigilante/assasin, and had plans to make her the new "Master" of the Knights of Aegon. However, things would not unfold this way. Gyllor learned from Rosaline Mormont that one of the surviving Targaryen children, Daenerys Targaryen, was ready to claim the Iron Throne, and she needed the Knights of Aegon now more than ever, particularly to help her fight the Sons of the Harpy in Meereen. Gyllor gave her his blessing, and Rosaline gave command of the Knights to Carmine Grayburn.