Even Japanese Prime Minister Shinzo Abe, who has shared the American skepticism of warming inter-Korean relations, greeted Kim Yong Nam.
They were seated not far from a section of North Korean cheerleaders who are attending the Olympics in a sign of warming ties between the two Koreas that Pence and other USA officials have warned against.
The sister of North Korean ruler Kim Jong Un attended the Pyeongchang Olympic Games opening ceremony on Friday.
North Korean leader Kim Jong Un's sister will watch the unified Korean ice hockey team play their first Olympic match on Saturday, the South Korean presidency said.
South Korea says North Korea has withdrawn a request that the South provide fuel to a North Korean ferry that transported more than a 100 artists for performances marking the Pyeongchang Winter Olympics.
U.S. Vice President Mike Pence is not directly addressing news that North Korean dictator Kim Jong Un has invited South Korean President Moon Jae-in for a visit, as the two Koreas use the Olympics as an opportunity for renewing ties. Athletes from the two countries marched side by side during the opening ceremony and have formed a joint women's hockey team.
At an earlier VIP reception for delegation leaders, Pence arrived late and stayed for just 5 minutes - and did not interact with the delegation from the North.
Providing fuel to the ferry would have been a hard decision for Seoul because of concerns that North Korea is trying to use the Olympics to poke holes in global sanctions against the country over its nuclear and missile programs.
But South Korea's presidential Blue House said Mr Pence had a meeting scheduled with United States athletes and had only planned to stay briefly to greet other officials.
The U.N. says Secretary-General Antonio Guterres had a brief exchange with North Korea's nominal head of state and "reiterated his expectation and hope that all parties will use dialogue to achieve the denuclearization of the Korean Peninsula".
Images from the lunch meeting at the Blue House, which was broadcast live on South Korean TV, showed Moon sat in front of Kim Yo Jong, rather than Kim Yong Nam, technically the more senior official present in the meeting and head of the North Korean delegation.
The officials toasted with the Korean liquor "soju" and were served buckwheat and potato pancakes, dried pollock soup, grilled halibut and other dishes.
Moon and Kim Yo Jong broke out broad smiles as they shook hands before the start of the opening ceremony.
It's unclear whether the occasion could be used to set up bigger meetings between the Koreas to ease tensions.
Pyongyang has ruled out a meeting with the USA delegation during the Winter Games.