Mr Turnbull stood by Mr Alexander, saying he "strictly complied" with the Clerk of the House's advice.
Precisely what Mr Alexander was attempting to convey by the joke is uncertain.
Earlier in the day, Ms Keneally was asked to justify why a man dressed as a chicken had confronted her Liberal opponent a day earlier holding Labor campaign material.
The party had been "energised and electrified" by the result, she added.
'I am here to claim success for the Labor movement.
She told Labor Party faithful, the seat is within reach at the next election. That means Labor could defeat the government in parliamentary votes if it has the backing of all five independents, or crossbenchers. The Liberal Party will retain the seat with a margin of 9.7 per cent, despite a swing against Alexander of around 5 per cent.
Labor's Kristina Keneally is challenging Liberal John Alexander.
"They have said 'yes John Alexander, you are Bennelong's champion, just as you have Australia's champion".
But the by-election has been less about the candidates, and more about the future of the Turnbull government and Malcolm Turnbull's leadership.
The final week of the campaign has been filled with stunts and dirty tricks from both sides as polls indicate the major parties are neck and neck.
"Labor ran their true campaign, a scare campaign, but I think not withstanding the people of Bennelong have seen through that", Senator Fierravanti-Wells told AAP.
'This is an unusual by-election in unusual circumstances, a swing certainly I saw it on the booths today'.
Australian Conservatives leader Cory Bernardi visited Epping Public School with candidate Joram Richa, where the pair were confronted by a transgender woman.
The event then took on a more subdued tone with many long faces starting at the screens.
Mr Turnbull acknowledged it was a "very, very close contest" when he and Mr Alexander met with voters at Gladesville Public School. Malcolm Turnbull, you are correct.
'Bill Shorten starts next year facing a potential four by because he didn't get his house in order'.
The byelection was triggered by the discovery Mr Alexander was a British citizen, in contravention of the Constitution.
Ms Keneally called the result "extraordinary" and insisted the by-election was "a big fight, but a fight worth having".
Mr Shorten told reporters the government was "out of touch, and here in Bennelong the people have a chance to say, "pull your socks up, mate".
'You can not translate a by-election result in Bennelong across the whole nation, ' Mr Pyne said.