Also a video has surfaced of a speech by Broadwell in which she suggested the Libya attack on September 11 was targeting a secret prison at the Benghazi consulate annex, raising unverified concerns about possible security leaks.
"I don't know if a lot of you have heard this, but the CIA annex had actually taken a couple of Libyan militia members prisoner and they think that the attack on the consulate was an effort to get these prisoners back," said Broadwell in a speech last month at the University of Denver.
Broadwell's source for that previously unpublished bit of information remains unclear, and there's no evidence so far that it came from Petraeus.
Administration officials have said the Benghazi assault was a terrorist attack.
Along with questions about Broadwell's access, the Petraeus' scandal also presents challenges to the congressional inquiry into the Benghazi attack.
Petraeus recently traveled to Libya to meet the CIA station chief to discuss the attack, CNN has confirmed. He was scheduled to testify before a congressional committee this week on the assault and the U.S. government response to it.
That now will not happen, but it is possible that he could be summoned by Congress to testify later.
Some Republicans have criticized the administration's response to the Benghazi attack and have speculated that Petraeus' departure was linked to the congressional inquiry.
Rep. Peter King, R-New York, said elements of the story "don't add up." He called Petraeus "an absolutely essential witness, maybe more than anybody else."
However, a senior U.S. official said Petraeus' departure wasn't connected to the Benghazi hearing.
"Director Petraeus' frank and forthright letter of resignation stands on its own," said the official, who spoke on the condition of anonymity because of the sensitivity of the topic. "Any suggestion that his departure has anything to do with criticism about Benghazi is completely baseless."
The Affair and the e-mails
Broadwell and Petraeus first met in 2006 at Harvard University's Kennedy School of Government, where she was a student, Broadwell wrote in the preface of the biography she co-authored on Petraeus.
She told him about her interest in studying military leadership, and he offered his help.
"I later discovered that he was famous for this type of mentoring and networking, especially with aspiring soldier-scholars," Broadwell wrote.
She traveled to Afghanistan, where she interviewed Petraeus repeatedly.
The actual affair began about two months after Petraeus took over at the CIA in September 2011, according to one Petraeus friend.
It ended about four months ago, and the two last talked about a month ago, the friend said.
The decision to end the relationship was mutual, the friend said
But at some point Broadwell also exchanged e-mails with Kelley.
The FBI joined the investigation when Kelley complained that Broadwell was sending harassing e-mails to her in May, a U.S. official told CNN.
According to a source with knowledge of the e-mails, the messages accused Kelley of untoward behavior with some generals at MacDill Air Force Base in Tampa, Florida where Kelley did volunteer work.
The e-mails detailed the "comings and goings of the generals and Ms. Kelley," said the source, who declined to speak on the record because of sensitivity of the investigation.
Among those believed to be referenced in the e-mails was Petreaus.
But now it seems that there are more, potentially damaging e-mails, that hold information about Allen, a four-star general who took over as the commander of the war effort in Afghanistan last year.
A job he took over from Petreaus.
™ & © 2012 Cable News Network, Inc., a Time Warner Company. All rights reserved.